Systems and methods for displaying medical imaging data

ABSTRACT

A system for displaying medical imaging data comprising one or more data inputs, one or more processors, and one or more displays, wherein the one or more data inputs are configured for receiving first image data generated by a first medical imaging device, wherein the first image data comprises a field of view (FOV) portion and a non-FOV portion, and the one or more processors are configured for identifying the non-FOV portion of the first image data and generating cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and transmitting the cropped first image data for display in a first portion of the display and additional information for display in a second portion of the display.

REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/775,622, filed Dec. 5, 2018, the entire contents of which are herebyincorporated by reference herein.

FIELD

The present disclosure relates generally to medical imaging, and moreparticularly to the processing of medical imaging for visualization oftissue.

BACKGROUND

With the advent of high-definition (HD) and Ultra HD/4K resolutions insurgical visualization, the 16:9 aspect ratio surgical display hasbecome increasingly common. However, many minimally-invasive surgicalprocedures still rely on an optical scope that results in a field ofview at the image sensor that is smaller than the image sensor sensingarea. This results in images and video having a circular field of viewarea within a substantial area of black pixels. In many cases, forexample, the utilization is only 44% of the usable imager area, whichcan result in utilization of only 44% of usable display area. This isespecially true for smaller diameter scopes, typically the 4 mm scopesused in arthroscopy and ENT/Neuro procedures.

When surgeons need to view multiple sources of information, such asmultiple low utilization optical scope imaging, the surgeons may have toeither switch the input on their primary surgical display between thevarious imaging feeds, use picture-in-picture or picture-by-picture modeon the surgical display, or look at two different monitors, which mightbe in two different locations in the operating room. All of theseoptions can cause either a sub-optimal usage of the viewable area of thesurgical display or cause the surgeon to context switch between focusingon one display to the other.

SUMMARY

According to some embodiments, medical imaging processing systems areconfigured to process and combine medical imaging data to generatedisplay feeds that provide enhanced display of medical imaging.According to some embodiments, the medical imaging processing systemscan combine multiple imaging data streams into one or more displaystreams for displaying data from multiple imaging sources and otherimaging session related information sources together in a single displaylayout. According to some embodiments, utilization of display layoutscan be optimized by removing unused portions of imaging data, such asdata generated by portions of an imager that are outside of a capturedfield of view. In some embodiments, display feeds can be generatedaccording to imaging session specific preferences tailored to specifictypes of imaging sessions and/or to specific imaging system users. Insome embodiments, a reconfigurable hardware processor of the medicalimaging processing system may be reconfigured from one imaging sessionto another to provide imaging data processing that is tailored to thenext imaging session. Through one or more of these capabilities, theimaging processing systems can provide enhanced medical imagingvisualization tailored to the preferences of the user.

According to some embodiments, a method of configuring a medical imagingprocessing system includes configuring a reconfigurable hardwareprocessor of the medical imaging processing system into a firstconfiguration for a first medical imaging session based on firstconfiguration data stored in a memory, wherein the first configurationimplements at least a first medical imaging processing algorithm;receiving first medical imaging data generated during the first medicalimaging session; generating enhanced first medical imaging data at leastin part by processing the first medical imaging data using the firstmedical imaging processing algorithm implemented in the firstconfiguration; displaying the enhanced first medical imaging data forobservation during the first medical imaging session; reconfiguring thereconfigurable hardware processor into a second configuration for asecond medical imaging session based on second configuration data storedin the memory, wherein the second configuration implements at least asecond medical imaging processing algorithm that is not implemented inthe first configuration; receiving second medical imaging data generatedduring the second medical imaging session; generating enhanced secondmedical imaging data at least in part by processing the second medicalimaging data using the second medical imaging processing algorithmimplemented in the second configuration; and displaying the enhancedsecond medical imaging data for observation during the second medicalimaging session on a display.

In any of these embodiments, the method may include receiving an inputindicative of the second medical imaging session and, in response toreceiving the input, automatically reconfiguring the reconfigurablehardware processor into the second configuration.

In any of these embodiments, the input may include a selection of a typeof medical procedure.

In any of these embodiments, the input may include a selection of a userprofile.

In any of these embodiments, the input may include selection of adefault configuration profile.

In any of these embodiments, the default configuration profile may bebased one or more connections to the medical imaging processing systemfrom one or more external devices.

In any of these embodiments, the default configuration profile may bebased on a field of view of a connected external device.

In any of these embodiments, the first configuration may be associatedwith a first type of medical procedure and the second configuration maybe associated with a second type of medical procedure.

In any of these embodiments, the first medical imaging session mayinclude performance of the first type of medical procedure on a patientand the second medical imaging session may include performance of thesecond type of medical procedure on the patient.

In any of these embodiments, the first configuration may be associatedwith a first user profile and the second configuration may be associatedwith a second user profile.

In any of these embodiments, the first medical imaging session mayinclude imaging a patient and the second medical imaging session mayinclude imaging the patient.

In any of these embodiments, the first configuration data and the secondconfiguration data may be both associated with the same type of medicalprocedure.

In any of these embodiments, the first medical imaging session may be afirst surgical session and the second medical imaging session may be asecond surgical session.

In any of these embodiments, the at least one medical imaging processingalgorithm implemented in the second configuration may include a smokedetection algorithm and generating the enhanced second medical imagingdata may include enhancing clarity of one or more portions of one ormore images associated with smoke.

In any of these embodiments, the first medical imaging processingalgorithm may be configured to detect a feature of imaged tissue.

In any of these embodiments, the feature of imaged tissue may be tissueperfusion, a location of a blood vessel, an amount of blood flow, adimension of imaged tissue, or a combination thereof.

In any of these embodiments, the enhanced second medical imaging datamay include an overlay on at least a portion of the second medicalimaging data.

In any of these embodiments, the reconfigurable hardware processor maybe reconfigured prior to a start of imaging.

In any of these embodiments, one or more medical imaging processingalgorithms may be implemented in both the first and secondconfigurations.

In any of these embodiments, the second medical imaging data may includeat least one of video frames and an image.

In any of these embodiments, the second medical imaging data may bereceived from an endoscopic imaging system.

In any of these embodiments, the second medical imaging data may bereceived from a camera control unit.

In any of these embodiments, the reconfigurable hardware processor maybe an FPGA or a GPU.

In any of these embodiments, the method may include receiving the secondmedical imaging data from a first device, receiving data from a secondmedical device, and outputting a display feed to the display, thedisplay feed comprising the enhanced second medical imaging data and atleast a portion of the data from the second medical device.

In any of these embodiments, the method may include receiving the secondmedical imaging data and the data from the second medical device at afirst processor, transmitting the second medical imaging data from thefirst processor to the reconfigurable hardware processor, receiving theenhanced second medical imaging data from the reconfigurable hardwareprocessor at the first processor, and generating, by the firstprocessor, the display feed by combining the enhanced second medicalimaging data with the at least a portion of the data associated with thesecond medical device.

In any of these embodiments, the first configuration data may be storedin a remote memory and received via a network connection.

According to some embodiments, a method for displaying medical imagingdata includes receiving first image data generated by a first medicalimaging device, wherein the first image data comprises a field of view(FOV) portion and a non-FOV portion; identifying the non-FOV portion ofthe first image data; generating cropped first image data by removing atleast a portion of the non-FOV portion of the first image data; anddisplaying the cropped first image data in a first portion of a displayand additional information in a second portion of the display.

In any of these embodiments, the non-FOV portion may be identified usingedge detection.

In any of these embodiments, the first image data may include a seriesof video frames and the edge detection may be performed on more than oneframe.

In any of these embodiments, the non-FOV portion may be identified usingone or more of a location of a center of the FOV portion and ameasurement associated with a dimension of the FOV portion.

In any of these embodiments, the location of a center of the FOV portionand the measurement associated with a dimension of the FOV portion maybe determined during an imaging session initialization process.

In any of these embodiments, the imaging session initialization processmay be a white balancing process.

In any of these embodiments, the first image data may include arectangular image or video frame and the FOV portion may be a circularportion of the rectangular image or video frame.

In any of these embodiments, the first image data may include a videoframe.

In any of these embodiments, the first image data may be received on afirst input of a medical imaging processing system and the additionalinformation may be based on data received on a second input of themedical imaging processing system.

In any of these embodiments, the method may include transmitting adisplay feed from the medical imaging processing system to the display,the display feed comprising a combination of the cropped first imagedata and the additional information.

In any of these embodiments, the method may include receiving secondimage data generated by a second medical imaging device; identifying anon-FOV portion of the second image data; generating cropped secondimage data by removing at least a portion of the non-FOV portion of thesecond image data; and displaying the cropped second image data in thesecond portion of the display.

In any of these embodiments, the first image data may be received on afirst input of a medical imaging processing system and the second imagedata may be received on a second input of the medical imaging processingsystem.

In any of these embodiments, the method may include transmitting adisplay feed from the medical imaging processing system to the display,the display feed including a combination of the cropped first image dataand the cropped second image data.

In any of these embodiments, the cropped first image data and theadditional information may be located on the display based onconfiguration data stored in a memory.

In any of these embodiments, the configuration data may includeuser-specified configuration data.

In any of these embodiments, the configuration data may be received viaa network connection.

In any of these embodiments, the first image data may be received froman endoscopic imaging system, an intraoperative C-arm imaging system, oran ultrasound system.

In any of these embodiments, the first image data may be received from acamera control unit.

In any of these embodiments, the additional information may include oneor more of patient data, metrics, a graph, an image, device status, anda video feed.

According to some embodiments, a reconfigurable medical imagingprocessing system includes a display; memory; a reconfigurable hardwareprocessor; and a second processor configured for: configuring thereconfigurable hardware processor into a first configuration for a firstmedical imaging session based on first configuration data stored in thememory, wherein the reconfigurable hardware processor in the firstconfiguration is configured to implement at least a first medicalimaging processing algorithm and to generate enhanced first medicalimaging data for display on the display at least in part by processingfirst medical imaging data using the first medical imaging processingalgorithm, and reconfiguring the reconfigurable hardware processor intoa second configuration for a second medical imaging session based onsecond configuration data stored in the memory, wherein thereconfigurable hardware processor in the second configuration isconfigured to implement at least a second medical imaging processingalgorithm and to generate enhanced second medical imaging data fordisplay on the display at least in part by processing second medicalimaging data using the second medical imaging processing algorithm.

In any of these embodiments, the second processor may be configured toreceive an input indicative of the second medical imaging session and,in response to receiving the input, automatically reconfigure thereconfigurable hardware processor into the second configuration.

In any of these embodiments, the input may include a selection of a typeof medical procedure.

In any of these embodiments, the input may include a selection of a userprofile.

In any of these embodiments, the input may include selection of adefault configuration profile.

In any of these embodiments, the default configuration profile may bebased on one or more connections to the medical imaging processingsystem from one or more external devices.

In any of these embodiments, the default configuration profile may bebased on a field of view of a connected external device.

In any of these embodiments, the first configuration may be associatedwith a first type of medical procedure and the second configuration maybe associated with a second type of medical procedure.

In any of these embodiments, the first medical imaging session mayinclude performance of the first type of medical procedure on a patientand the second medical imaging session may include performance of thesecond type of medical procedure on the patient.

In any of these embodiments, the first configuration may be associatedwith a first user profile and the second configuration may be associatedwith a second user profile.

In any of these embodiments, the first medical imaging session mayinclude imaging a patient and the second medical imaging session mayinclude imaging the patient.

In any of these embodiments, the first configuration data and the secondconfiguration data may be both associated with the same type of medicalprocedure.

In any of these embodiments, the first medical imaging session may be afirst surgical session and the second medical imaging session may be asecond surgical session.

In any of these embodiments, the at least one medical imaging processingalgorithm implemented in the second configuration may include a smokedetection algorithm and generating the enhanced second medical imagingdata may include enhancing clarity of one or more portions of one ormore images associated with smoke.

In any of these embodiments, the first medical imaging processingalgorithm may be configured to detect a feature of imaged tissue.

In any of these embodiments, the feature of imaged tissue may be tissueperfusion, a location of a blood vessel, an amount of blood flow, adimension of imaged tissue, or a combination thereof.

In any of these embodiments, the enhanced second medical imaging datamay include an overlay on at least a portion of the second medicalimaging data.

In any of these embodiments, the system may be configured to reconfigurethe reconfigurable hardware processor prior to a start of imaging.

In any of these embodiments, one or more medical imaging processingalgorithms may be implemented in both the first and secondconfigurations.

In any of these embodiments, the second medical imaging data may includeat least one of video frames and an image.

In any of these embodiments, the system may be configured to receive thesecond medical imaging data from an endoscopic imaging system.

In any of these embodiments, the system may be configured to receive thesecond medical imaging data from a camera control unit.

In any of these embodiments, the reconfigurable hardware processor maybe an FPGA or a GPU.

In any of these embodiments, the system may be configured to receive thesecond medical imaging data from a first device, receive data from asecond medical device, and display the enhanced second medical imagingdata and at least a portion of the data from the second medical device.

In any of these embodiments, the system may be configured to receive thesecond medical imaging data and the data from the second medical deviceat the second processor, transmit the second medical imaging data fromthe second processor to the reconfigurable hardware processor, receivethe enhanced second medical imaging data from the reconfigurablehardware processor at the second processor, and generate, by the secondprocessor, a display feed for the display by combining the enhancedsecond medical imaging data with the at least a portion of the dataassociated with the second medical device.

In any of these embodiments, the first configuration data may be storedin a remote memory and received via a network connection.

According to some embodiments, a system for displaying medical imagingdata includes one or more data inputs; one or more processors; and oneor more displays, wherein the one or more data inputs are configured forreceiving first image data generated by a first medical imaging device,wherein the first image data comprises a field of view (FOV) portion anda non-FOV portion, and the one or more processors are configured foridentifying the non-FOV portion of the first image data and generatingcropped first image data by removing at least a portion of the non-FOVportion of the first image data, and transmitting the cropped firstimage data for display in a first portion of the display and additionalinformation for display in a second portion of the one or more displays.

In any of these embodiments, the one or more processors may beconfigured for identifying the non-FOV portion using edge detection.

In any of these embodiments, the first image data may include a seriesof video frames and the one or more processors may be configured foridentifying the non-FOV portion using edge detection performed on morethan one frame.

In any of these embodiments, the one or more processors may beconfigured for identifying the non-FOV portion using one or more of alocation of a center of the FOV portion and a measurement associatedwith a dimension of the FOV portion.

In any of these embodiments, the one or more processors may beconfigured for determining the location of a center of the FOV portionand the measurement associated with a dimension of the FOV portionduring an imaging session initialization process.

In any of these embodiments, the imaging session initialization processmay be a white balancing process.

In any of these embodiments, the first image data may include arectangular image or video frame and the FOV portion may be a circularportion of the rectangular image or video frame.

In any of these embodiments, the first image data may include a videoframe.

In any of these embodiments, the one or more data inputs may beconfigured for receiving the first image data on a first input of amedical imaging processing system and the additional medical imagingdata may be based on data received on a second input of the medicalimaging processing system.

In any of these embodiments, the medical imaging processing system maybe configured for transmitting a display feed from the medical imagingprocessing system to the display, the display feed may include acombination of the cropped first image data and the additional medicalimaging data.

In any of these embodiments, the one or more data inputs may beconfigured for receiving second image data generated by a second medicalimaging device; and the one or more processors may be configured for:identifying a non-FOV portion of the second image data, generatingcropped second image data by removing at least a portion of the non-FOVportion of the second image data, and transmitting the cropped secondimage data for display in a second portion of the one or more displays.

In any of these embodiments, the one or more data inputs may beconfigured for receiving the first image data on a first input of amedical imaging processing system and receiving the second image data ona second input of the medical imaging processing system.

In any of these embodiments, the medical imaging processing system maybe configured for transmitting a display feed from the medical imagingprocessing system to the display, the display feed comprising acombination of the cropped first image data and the cropped second imagedata.

In any of these embodiments, the cropped first image data and theadditional medical imaging data may be located on the display based onconfiguration data stored in a memory.

In any of these embodiments, the configuration data may includeuser-specified configuration data.

In any of these embodiments, the system is configured for receiving theconfiguration data via a network connection.

In any of these embodiments, the one or more data inputs may beconfigured for receiving the first image data from an endoscopic imagingsystem, an intraoperative C-arm imaging system, or an ultrasound system.

In any of these embodiments, the one or more data inputs may beconfigured for receiving the first image data from a camera controlunit.

In any of these embodiments, the additional information may include oneor more of patient data, metrics, a graph, an image, device status, anda video feed.

According to some embodiments, a non-transitory tangiblecomputer-readable medium includes computer-executable program codeembedded thereon to perform the any of the methods above.

According to some embodiments, a kit for processing a time series offluorescence images of tissue of a subject includes any of the systemsabove and/or any of the non-transitory tangible computer-readable mediumabove, and a fluorescence imaging agent.

According to some embodiments, a fluorescence imaging agent is providedfor use in any of the methods above, in the any of the systems above, orin any of the kits above for imaging an object.

In any of these embodiments, imaging an object may include imaging anobject during blood flow imaging, tissue perfusion imaging, lymphaticimaging, or a combination thereof.

In any of these embodiments, blood flow imaging, tissue perfusionimaging, and/or lymphatic imaging may include blood flow imaging, tissueperfusion imaging, and/or lymphatic imaging during an invasive surgicalprocedure, a minimally invasive surgical procedure, or during anon-invasive surgical procedure.

In any of these embodiments, the invasive surgical procedure may includea cardiac-related surgical procedure or a reconstructive surgicalprocedure.

In any of these embodiments, the cardiac-related surgical procedure mayinclude a cardiac coronary artery bypass graft (CABG) procedure.

In any of these embodiments, the CABG procedure may be on pump or offpump.

In any of these embodiments, the non-invasive surgical procedure mayinclude a wound care procedure.

In any of these embodiments, the lymphatic imaging may includeidentification of a lymph node, lymph node drainage, lymphatic mapping,or a combination thereof.

In any of these embodiments, the lymphatic imaging may relate to thefemale reproductive system.

Some embodiments include use of any of the methods above in any of thesystems above or in any of the kits above for imaging an object forlymphatic imaging.

Some embodiments include use of any of the methods above, in any of thesystems above, or in any of the kits above for imaging an object forblood flow imaging, tissue perfusion imaging, or a combination thereof.

It will be appreciated that any variations disclosed herein inconnection with the methods, systems, kits and other aspects of thedisclosure may be may be combined.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described, by way of example only, withreference to the accompanying drawings, in which:

FIG. 1 is a block diagram of a system for generating and displayingmedical imaging data during medical imaging sessions, according to someembodiments;

FIG. 2 illustrates a method for displaying medical imaging data,according to some embodiments;

FIG. 3A illustrates an exemplary image generated by an endoscopic imagerand FIG. 3B illustrates two endoscopic images displayed side-by-side onan exemplary display;

FIG. 3C illustrates an exemplary display, according to some embodiments,displaying cropped endoscopic images, according to some embodiments;

FIG. 3D illustrates an exemplary display, according to some embodiments,displaying cropped endoscopic images and additional imaging sessionrelated data, according to some embodiments;

FIG. 4 is a block diagram of a medical imaging data processing hub,according to some embodiments;

FIG. 5A illustrates an example of a first predefined display layout thatcan be generated by the hub of FIG. 4, and FIG. 5B illustrates anexample of a second predefined display layout that can be generated bythe hub of FIG. 4;

FIG. 6 illustrates an example of a medical imaging processing hubconfigured for a first imaging session, according to some embodiments;

FIG. 7 illustrates a method for configuring a medical imaging processingsystem, according to some embodiments;

FIGS. 8A and 8B are block diagrams of a medical imaging processingsystem performing the method of FIG. 7, according to one embodiments;

FIGS. 9A and 9B illustrate graphical user interfaces for configuring amedical imaging processing system for a new imaging session, accordingto some embodiments;

FIG. 10 is an illustrative depiction of an exemplary fluorescenceimaging system, according to some embodiments;

FIG. 11 is an illustrative depiction of an exemplary illumination moduleof a fluorescence imaging system, according to some embodiments;

FIG. 12 is an exemplary camera module of a fluorescence imaging system,according to some embodiments; and

FIG. 13 is an exemplary endoscopic imaging cart, according to someembodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to implementations and embodimentsof various aspects and variations of systems and methods describedherein. Although several exemplary variations of the systems and methodsare described herein, other variations of the systems and methods mayinclude aspects of the systems and methods described herein combined inany suitable manner having combinations of all or some of the aspectsdescribed. Described herein are systems and methods for generatingenhanced medical imaging for display in connection with (e.g., during) amedical imaging session. Systems and methods can process data frommultiple imaging systems to generate enhanced imaging and can stitchtogether multiple imaging data sets into single display feeds fordisplaying information from multiple sources on a single display.Imaging data can be processed to maximize utilization of displays toenable the presentation of more relevant information to the practitionerduring the imaging session.

According to some embodiments, the systems and methods can process andcombine imaging data differently based on the needs of each imagingsession. Practitioners may be able to define the information that isdisplayed during their imaging sessions, ensuring that data is presentedin a manner suited to the practitioner, which can reduce the amount oftime needed for the practitioner to adjust data display.

In some embodiments, one or more reconfigurable hardware processors arereconfigured for each imaging session to provide imaging processing thatis tailored to each imaging session. Reconfigurable hardware processors,such as field-programmable gate arrays (FPGA's) provide the low latencyand high bandwidth required for real time video processing and alsoprovide for the ability to implement different algorithms or differentcombinations of algorithms on different data inputs or combinations ofdata inputs as required from imaging session to imaging session,providing for imaging processing that is tailored to meet the differingneeds of different imaging sessions. This configurability andflexibility in the ability to process and combine different input dataenables a single imaging processing system, according to embodimentsdescribed herein, to support a wide variety of imaging sessions,including a wide variety of surgical procedures.

In the following description of the various embodiments, reference ismade to the accompanying drawings, in which are shown, by way ofillustration, specific embodiments that can be practiced. It is to beunderstood that other embodiments and examples can be practiced, andchanges can be made without departing from the scope of the disclosure.

In addition, it is also to be understood that the singular forms “a,”“an,” and “the” used in the following description are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It is also to be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It is further to beunderstood that the terms “includes, “including,” “comprises,” and/or“comprising,” when used herein, specify the presence of stated features,integers, steps, operations, elements, components, and/or units but donot preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, units, and/or groupsthereof.

Certain aspects of the present disclosure include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present disclosurecould be embodied in software, firmware, or hardware and, when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by a variety of operating systems. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that, throughout the description, discussionsutilizing terms such as “processing,” “computing,” “calculating,”“determining,” “displaying,” “generating” or the like, refer to theaction and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system memories orregisters or other such information storage, transmission, or displaydevices.

The present disclosure in some embodiments also relates to a device forperforming the operations herein. This device may be speciallyconstructed for the required purposes, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina non-transitory, computer readable storage medium, such as, but notlimited to, any type of disk, including floppy disks, USB flash drives,external hard drives, optical disks, CD-ROMs, magnetic-optical disks,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, application specific integratedcircuits (ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The methods, devices, and systems described herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct amore specialized apparatus to perform the required method steps. Therequired structure for a variety of these systems will appear from thedescription below. In addition, the present invention is not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the present invention as described herein.

FIG. 1 illustrates a system 100 for generating and displaying medicalimaging data during medical imaging sessions. System 100 includes amedical data processing hub 102 that processes data received from one ormore imaging modalities 104 to generate one or more display feeds fordisplaying enhanced medical imaging on one or more displays 106. The oneor more imaging modalities 104 may generate image data associated withtreatment of a patient. The image data can be images or videos generatedduring treatment of the patient in support of one or more medicalprocedures, such as video captured by an endoscopic camera during anendoscopic procedure on a patient. Examples of medical imagingmodalities include, without limitation, endoscopic systems, open fieldimaging systems, x-ray systems such as intraoperative c-arm systems,computer tomography systems, ultrasound systems, magnetic resonanceimaging systems, and nuclear medicine systems.

In some embodiments, hub 102 may receive data from one or morenon-imaging devices 120 that may be used in connection with (e.g.,during) a medical imaging session and that may provide information thatmay be relevant for display during a medical imaging session.Non-limiting examples of non-imaging devices include insufflators,illumination controllers, and voice control systems.

Hub 102 may receive image data from the one or more imaging modalities104 through one or more input ports 108. The hub 102 generates one ormore display feeds using received imaging data and transmits the one ormore display feeds to one or more displays 106 via one or more outputports 110. For example, the hub 102 may generate a display feed thatincludes enhanced imaging of tissue of a patient based on imaginggenerated by one or more imaging modalities 104 and the enhanced imagingmay be displayed on one or more of the displays 106 to assist apractitioner during treatment of the patient. Hub 102 may also transmitdisplay feeds to one or more recording devices 112 for recordingenhanced imaging for later retrieval. Input ports 108 and output ports110 may be any suitable types of data transmission ports, such as DVIports, HDMI ports, RS232 ports, IP ports, and the like.

Hub 102 may be connected to one or more networks 116 via one or morenetwork connections 118. The one or more networks may be local networksuch as a hospital information system or may be a wider network such asa wide area network or the internet. A network connection 118 can be awired connection, such as an Ethernet connection, or a wireless networkconnection, such as a Wi-Fi connection. In some embodiments, the hub 102may access the one or more networks 116 to retrieve configuration datastored at a network location for configuring the hub for an imagingsession, and/or may access the one or more networks to receive updatedsoftware and/or updated hardware files for processing imaging data.

One or more user interfaces 114 may be connected to hub 102 for a userto provide input to the hub 102. The user may input data related forconfiguring the hub 102 for an imaging session. User input can include,for example, selection of a practitioner profile associated with anupcoming imaging session, selection of a type of imaging session ortypes of procedure to be performed during an imaging session, or anyother relevant information. The one or more user interfaces 114 mayinclude a tablet, a keyboard, a mouse, a voice control system, a keypad,a touchscreen, or any combination thereof.

As described in detail below, the hub 102 processes received medicalimaging data and any other relevant data and generates enhanced displayfeeds for display on one or more displays 106 during an imaging session.According to some embodiments, the hub 102 can combine multiple imagingsources into a single display feed, can process received imaging data togenerate richer imaging data, can modify imaging data for betterutilization of display space, and/or can reconfigure the processing ofimaging data depending on the needs and preferences of users fromimaging session to imaging session.

FIG. 2 illustrates a method 200 for displaying medical imaging data,according to some embodiments. Method 200 may be performed by a medicalimaging data processing hub, such as medical imaging data processing hub102 of system 100. Method 200 is performed for removing unutilizedportions of received imaging data to better utilize display space, whichcan provide the ability to display more relevant information to usersduring an imaging session.

In many conventional imaging systems such as scope-based imagingsystems, including for example endoscopic imaging systems, a generallycircular area light from the scene is projected on the light-sensitiveportions of the imaging sensor or sensors. This is due to the sensor orsensors of the imager being generally larger in area than the area oflight provided by the scope optics. Therefore, the imaging captured bythe sensor or sensors includes a field of view (FOV) portionrepresenting the light received from the field of view and a non-FOVportion generated by portions (i.e., pixels) of the sensor or sensorsthat do not receive light from the scene, often resulting in arectangular image having a circular FOV portion in the middle that showsthe imaged scene surrounded by black non-FOV portions (or near black dueto sensor noise). When the endoscopic imaging is displayed in aconventional manner, a large portion of the display is taken up by thenon-FOV portion, which displays black pixels that do not provide anyuseful information.

To illustrate this concept, an imaging system such as an exemplaryendoscopic image 300 is shown in FIG. 3A. Image 300 includes a FOVportion 302 generated by portions of the sensor receiving light from theimaged scene and a non-FOV portion 304 generated by portions of thesensor that do not receive light from the scene. FIG. 2B illustrates twoimages 300 displayed side-by-side on an exemplary display 350. Asillustrated, a relatively large amount of display space is wasted due tothe non-FOV portions of the two images. In some embodiments, the hub 102can crop some or all of the non-FOV portion of received image data.

Returning to FIG. 2, at step 202, first image data that has beengenerated by a first medical imaging device is received by the medicalimaging data processing hub. The first image data, which can be an imageor video frames, includes an FOV portion and a non-FOV portion. Forexample, the first image data may be a video frame generated by anendoscopic camera, such as image 300 of FIG. 3A. The frame may includean FOV portion generated by pixels of the camera sensor or sensors thatreceive light from an imaged scene incident on the sensor or sensors andmay include a non-FOV portion generated by pixels of the camera sensoror sensors that do not receive light from the imaged scene.

At step 204, the non-FOV portion of the first image data is identified.According to some embodiments, the non-FOV portion may be identifiedbased on one or more predetermined parameters associated with the FOVportion. Examples of predetermined parameters include a center of theFOV portion, a radius or diameter of the FOV portion, and pixellocations associated with the FOV portion or non-FOV portions. Pixelsthat are outside of an area defined by the predetermined parameters maybe identified as the non-FOV portion.

In some embodiments, parameters associated with an FOV portion of datareceived from a connected device may be determined once and usedrepeatedly as new image data is received from the device to identify thenon-FOV portions of the data received from the connected device. Forexample, a center and diameter (or radius) of a FOV portion may bedetermined based on image data received from a connected device and thiscenter and diameter (or radius) may be used to identify the non-FOVportion in future image data received from the device. In someembodiments, one or more edge detection algorithms are used to detectthe edge of the FOV portion and the edge data may be used to identifynon-FOV portions of an image or may be used to determine the center anddiameter (or radius) of the FOV portion, which in turn, is used toidentify non-FOV portions of an image.

In some embodiments, one or more parameters associated with a FOVportion of image data received from a connected device are determinedduring an imaging session initialization phase of the connected devicein which images are captured that have a sharp boundary between the FOVportion and the non-FOV portion. This initialization phase may be, forexample, a white balance phase in which an imager is directed toward awhite surface to allow for the imager and/or associated light source toadjust one or more imaging parameters such as gain and light intensitybased on an amount of light received from the white surface. During thewhite balance phase, the FOV portion of image data generated by theimager, which is directed at a white background, is relatively brightand, as such, has high contrast with the non-FOV portion, which isblack, providing a clear edge that can be readily detected using one ormore edge detection algorithms.

In some embodiments, the medical imaging data processing hub receives anindication from a connected device that the connected device is in aninitialization phase, such as a white balance phase. In response toreceiving this indication, the medical imaging data processing hubperforms an edge detection process to determine the location of the FOVportion of the image data received from the connected device. Thedetermined location of the FOV portion (e.g., center, diameter, pixellocations, etc.) can be used to identify the non-FOV portion of imagedata subsequently received.

In some embodiments, the non-FOV portion may be identified by detectingthe location of the perimeter of the FOV portion for each received imageor frame. In some embodiments, the perimeter of the FOV portion in thefirst image data may be detected using, for example, one or more edgedetection algorithms.

At step 206, cropped first image data is generated by removing at leasta portion of the non-FOV portion of the first image data. The non-FOVportion or portions that are removed may be selected based on anysuitable cropping criteria, including a desired aspect ratio for acropped image or a predefined size of a cropped image. For example, acropping criteria may specify that the cropped image should be squareand, based on this criteria, non-FOV portions that are outside of asquare encompassing the FOV portion may be removed, resulting in asquare cropped image. Alternatively, a cropping criteria may specify anaspect ratio and, based on this criteria, non-FOV portions of arectangle encompassing the FOV portion may be removed, resulting in acropped image having the specified aspect ratio.

In some embodiments, one or more cropping criteria used in step 206 maybe based on one or more properties of a connected display. For example,the dimensions of the display may be used to determine the bounds of thecropped image. The display dimensions may be divided into displaysections and the dimensions of the display sections may determine thebounds of a cropped image. For example, in the exemplary display of FIG.2D, the first display section 220 may be sized such that the image fordisplay in section 220 may be cropped to the width of the FOV portion ofthe image for display in section 220, whereas an image for display inthe second section 222 may be cropped to the height of the FOV portionof the image for display in section 222.

In some embodiments, the medical imaging data processing hub may receiveinformation regarding the display area (i.e., pixel dimensions, spatialdimensions, etc.) from the connected display. In other embodiments, oneor more display area parameters are user defined.

At step 208, a display feed is generated based on the cropped firstimage data. The display feed is transmitted to one or more connecteddisplays such as display 106 of system 100, via one or more displayconnections, and the cropped first image data is displayed on a display.In some embodiments, the cropped first image may be displayed in a firstportion of the display and additional information may be displayed in asecond portion of the display. Examples of additional information thatmay be displayed include one or more images, videos, patient data and/orpatient metadata, connected device status, metrics associated withimaging or any other connected device related information, one or moregraphs, etc. According to some embodiments, by cropping the first imagedata, the first portion of the display may occupy less room on thedisplay, increasing an amount of display space available for displayingthe additional information. According to some embodiments, the croppedimage data may be shown in a portion of the screen having a same heightand/or width as would have a portion showing uncropped image data, butcropping of the image data allows for the FOV portion to be larger onthe display.

In some embodiments, image data may be received from a plurality ofconnected devices and the image data from each connected device may becropped according to method 200 discussed above. A display feed may begenerated for displaying the multiple cropped images on one or moreconnected displays. In some embodiments, additional information may bedisplayed along with one or more cropped images. The additionalinformation may be based on data received from one or more connecteddevices. For example, an insufflator system connected to the medicalimaging data processing hub may transmit an insufflation pressurereading to the system and the pressure reading may be combined with acropped endoscopic image in the display feed for displaying alongsidethe cropped endoscopic image on the display.

FIG. 3C illustrates the exemplary display 350 of FIG. 3B with twocropped images 310 generated according to method 200. The cropped images310 include the FOV portions 302 of the images 300 of FIG. 3B withportions of the non-FOV portions removed. As illustrated, cropping ofthe images allows for the images to be displayed much larger. Croppingof images can also provide space for additional information to bedisplayed. For example, in FIG. 3C, the cropped image 310 occupies afirst portion 320 of the display 350, a second cropped image 326occupies a second portion 322 of the display 350, and an exemplary graph328 occupies a third portion 324 of the display screen. Thus, a medicalimaging processing system, such as hub 102, is able to maximizeutilization of a display screen for displaying medical imaging data andother information.

FIG. 4 is a block diagram of a medical imaging data processing hub 400,according to one embodiment, that may be used in a medical imagingsystem, such as system 100 of FIG. 1, to process multiple data streamsfrom connected medical devices, such as imaging devices, and generate anoptimized display layout for displaying useful information to a user,such as a surgeon, during a medical procedure. Hub 400 includes one ormore input connections 402 for receiving data from connected devices.Hub 400 includes one or more outputs 404 for connection to one or moredisplay devices. Hub 400 includes a primary processing unit 406 thatprocesses at least a portion of the data received from connected devicesand generates a display feed for outputting to one or more connecteddisplays.

The hub 400 includes a primary processing unit 406 for managingprocessing of imaging data and generating display feeds using processeddata, a reconfigurable hardware processor 408 for processing imagingdata streams, and an auxiliary processing unit 410 for providingsoftware-based processing of imaging data and other data.

The reconfigurable hardware processor 408 may be a Field ProgrammableGate Array (FPGA) that can be reconfigured by loading hardware logicfiles that define circuit connections in the FPGA. The reconfigurablehardware processor 408 provides low latency and high bandwidthprocessing of imaging data and can be repeatedly reconfigured to providedifferent processing of imaging data for different imaging sessions. Byleveraging a reconfigurable hardware processor, the hub 400 can provideenhanced imaging data, such as video, in real time with little or nodelay between capture of imaging and display of enhanced imaging on aconnected display during an imaging session. In some embodiments, thereconfigurable hardware processor 408 is a reconfigurable GPU. Theprimary processing unit 406 and auxiliary processing unit 410 may eachbe any suitable processor or combinations of processors, such as acentral processing unit, a graphics processing unit, a microcontroller,an ASIC, or an FPGA any combination thereof.

The hub 400 includes memory 412, which may be a local memory locatedwithin hub 400 or may be a remote memory in a remote location that hub400 can access through a network connection. One or more portions ofmemory 412 may be local and one or more portions may be remotelylocated. Memory 412 may include one or more configuration files 414 thatspecify configurations for the hub 400 for different imaging sessions,one or more software programs for executing by the primary processingunit 406 and/or the auxiliary processing unit 410, and one or morehardware logic files 418 for reconfiguring the reconfigurable hardwareprocessor 408. The primary processing unit 406 may access aconfiguration file 414 to determine the processing requirementsspecified in the configuration file, may load a hardware logic file 418onto the reconfigurable hardware processor 408 as defined by theconfiguration file, and may load a software program 416 onto theauxiliary processing unit 410 as specified in the configuration file414. Thus, the data stored in memory 412 can be used to configure thehub 400 for different imaging sessions.

The reconfigurable hardware processor 408 is communicatively connectedto the primary processing unit 406. The primary processing unit 406 maysend video data streams to the reconfigurable hardware processor 408 forprocessing and may receive processed video back from the reconfigurablehardware processor 408 for inclusion in a display feed. The primaryprocessing unit may load hardware logic files to the reconfigurablehardware processor 408 for reconfiguring the reconfigurable hardwareprocessor 408.

The auxiliary processing unit 410 is communicatively coupled to theprimary processing unit 406. The primary processing unit 406 may senddata to the auxiliary processing unit 410 for processing and may receivethe results of the processing for inclusion in a display feed. Theprimary processing unit 406 may load software on the auxiliaryprocessing unit 410 for processing imaging data.

Hub 400 is configured to combine information received from multipleconnected devices into a display feed for display on a connecteddisplay. Accordingly, multiple information sources can be displayedsimultaneously on the connected display. Hub 400 is configured to stitchtogether information received from connected devices according topredefined layouts. For example, hub 400 may generate a display feed inwhich a first video stream is displayed in a first display section, asecond video stream is displayed in a second display section, andadditional information, such as data, alerts, device status, metrics,etc., is displayed in a third display section.

According to some embodiments, the primary processing unit 406 isresponsible for receiving data from connected devices and stitching thedata together into a composite display feed. The primary processing unit406 may leverage the reconfigurable hardware processor 408 and/or theauxiliary processing unit 410 to process received data for enhancingdisplay of the data.

The primary processing unit 406 combines information sources into adisplay feed according to one or more predefined display layouts thatspecify the types of imaging information to be displayed and therelative sizes and locations of imaging information and otherinformation for display. Predefined display layouts may be associatedwith different types of imaging sessions, such as different types ofsurgical sessions or different types of surgical or other medicalprocedures. Different types of procedures may involve different types ofimaging equipment and/or different types of imaging processingalgorithms, and the predefined display layouts may specify the types ofinformation for display for a given procedure. Predefined displaylayouts may be associated with different practitioners according to thepreferences of the practitioners. For example, the same information maybe displayed in different ways for two different practitionersperforming the same procedures. Predefined display layouts may be storedas configuration data files 414 in memory 412.

FIG. 5A illustrates an example of a first predefined display layout 500and FIG. 5B illustrates an example of a second predefined display layout520. The first layout 500 includes three sections for three differentsources-502, 504, and 506. The term source refers to a distinct dataoutput generated by the hub 400. Sources can include data received fromone or more connected devices, enhanced data generated by processingdata received from one or more connected devices, or any combinationthereof. Multiple sources can include or be based on the same datareceived from a connected device. For example, a first source caninclude a video stream received from a connected device and a secondsource can include the same video stream enhanced with informationextracted from the video stream or information received from anotherconnected device.

In addition to defining the sources to be displayed, predefined displaylayouts define the relative locations of the different sources on thedisplay and the relative sizes of the different sources on the display.For example, in first layout 500 the first source 502 is located abovethe second source 504 on the left half of the display, with both sources502 and 504 being equal in size. The third source 506 is located on theright half of the display and is larger than the first and secondsources. In contrast, layout 520 includes six different sources of equalsize arranged in two rows of three columns. Layout 520 includes thefirst, second, and third sources 502, 504, and 506, in addition to threeother sources. The first and second sources 502 and 504 are in differentlocations relative to layout 500 and the third source 506 is a differentsize relative to layout 500. Layout 500 may be associated with a firstpractitioner who configured the layout 500 according to his or herpreferences and layout 520 may be associated with a second practitioner.Layouts 500 and 520 may be associated with different types of imagingsessions, such as different types of surgical procedures, or may beassociated with the same type of surgical procedure. In someembodiments, both layouts are used in the same imaging session. Forexample, layout 500 may define the layout for a first display of animaging system and layout 520 may define the layout for a second displayof the imaging system.

The hub 400 may configure a display feed according to one or moreparameters associated with an imaging session. Hub 400 may be used formultiple different types of medical procedures and/or by multipledifferent practitioners. As used herein, a medical procedure may referto a single (e.g., operative) procedure with various tasks beingperformed by the practitioner (e.g., surgeon) or to more than oneprocedure being performed in a single session with a patient (e.g., asingle operating session on a patient). For example, an orthopedicoperating session involving performing orthopedic procedures (e.g.,drilling and/or implantation of medical devices) along with an imagingprocedure (e.g., to visualize the tissue space and/or blood flow/tissueperfusion) may be a single medical procedure or may be multiple medicalprocedures. Different types of medical procedures may utilize differenttypes of imagers and other equipment. Display layouts designed for onetype of procedure may not be as suitable for another type of procedure.Furthermore, different practitioners may have different preferences withregard to what type of information should be displayed and how theinformation is displayed. Accordingly, hub 400 may process received dataand generate display feeds differently based on the specific requirementor preferences of each medical imaging session.

Hub 400 may configure processing of input data and generation of displayfeeds based on one or more predefined configurations. Predefinedconfigurations may be associated with one or more parameters of animaging session. Examples of imaging session parameters can include user(e.g., practitioner), procedure type, information associated with one ormore connected input devices, and information associated with one ormore connected output devices.

The hub 400 may receive user input (such as through user interface 114of FIG. 1) specifying one or more parameter values and may select apredefined configuration based on the one or more parameter values. Thehub 400 reconfigures processing of one or more inputs and generation ofone or more display feeds based on the selected predefinedconfiguration.

Predefined configurations may define predefined display layouts and mayalso define one or more data processing algorithms. Algorithms may beimplemented, for example, in reconfigurable hardware processor 408and/or in auxiliary processing unit 410. In some embodiments, thereconfigurable hardware processor 408 may be reconfigured according tothe predefined configuration in order to perform imaging data processingthat is specified by the predefined configuration.

As explained above, different sources can be included in differentlayouts. Different sources can be data from different connected devices,but can also be different information extracted from the same connecteddevices. To facilitate generation of different data depending on theconnected devices and the layout preferences from one imaging session tothe next, hub 400 may automatically reconfigure the processing of datareceived from connected devices according to the requirement specifiedin the configuration data associated with an imaging session.

Hub 400 may receive an indication of an imaging session that isassociated with a predefined configuration and may automaticallyconfigure processing of input data and generation of display feedsaccordingly. For example, in preparation for a surgical session, a nursemay input one or more parameters associated with the surgical session tothe hub 400, such as through a keyboard, mouse, touchscreen, or otherinput device, and the hub 400 may configure itself accordingly, whichmay include reconfiguring the reconfigurable hardware processor byloading one or more hardware logic files stored in memory 412 andloading one or more software programs or modules on auxiliary processingunit 410. Parameters can include the type of surgery to be performed andthe practitioner performing the surgery. One or more predefined layoutsmay be associated with the type of surgery and/or the practitioner andthe hub 400 may reconfigure itself to generate a display feed accordingto the predefined layout.

FIG. 6 illustrates an example of a medical imaging processing hub, suchas a hub 400, configured for a first imaging session. Configured hub 600includes a primary processor 602, a reconfigurable hardware processor604, and an auxiliary processor 606. Hub 600 includes multiple datainputs 608, three of which are connected to three different devices(630, 632, and 634), which can be cameras, camera control units,instrument control units, lighting control units, insufflators,cauterizers, or any other device or system used during the imagingsession that generates data relevant to the imaging session. Hub 600includes multiple video outputs 610. In the illustrated embodiment, twodisplays 640 and 642 are connected to two of the video outputs 610.

In the first configuration, one or more algorithms have been loaded intothe reconfigurable hardware processor 604 for processing imaging datareceived from device 632. The reconfigurable hardware processor 604processes data received from device 632 and transmits the processed datato the primary processor 602. The reconfigurable hardware processor 604may receive imaging data directly from the input 608 or may receive datavia the primary processor 602. In some embodiments, the primaryprocessor 602 may crop image data, according to the principles describedabove with respect to method 200 of FIG. 2, and provide the croppedimage data to the reconfigurable hardware processor 604 and/or to theauxiliary processor 606. This may be advantageous in reducing the amountof imaging data requiring processing.

The auxiliary processor 606 executes a software based program forprocessing data from the third connected device 634. The auxiliaryprocessor 606 may output the results of the processing to the primaryprocessor 602 via, for example, a video output 612, such as a videooutput on the mother board for the CPU.

The primary processor 602 is responsible for combining the differentdata sources into a display feed for transmission to the connecteddisplay 640. The primary processor stiches together the processed datafrom the reconfigurable hardware processor 604, from the auxiliaryprocessor 606, and directly from a first connected device 630. Forexample, the primary processor may generate a display feed that locatesthese three sources into different sections of the display.

In some embodiments, the primary processor 602 may be configured toprovide multiple different display streams. In the illustratedembodiment, the primary processor 602 includes two compositors 614 and616 that can generate two different display feeds, according to, forexample, configuration data stored in memory. The first compositor 614is configured to combine the data from the first connected device, thereconfigurable hardware processor 604, and the auxiliary processor 606into a first display feed for transmission to display 640. The secondcompositor 616 receives data input (e.g., video input) from device 634and generates a second display feed for transmission to display 642.

The compositors may combine data sources and generate display feedsdifferently from one imaging session to the next. The handling of databy the compositors may be altered based on configuration data stored inmemory. For example, the data handling illustrated in FIG. 6A may bedefined by a first configuration file. A second configuration file mayspecify alteration of this data handling by, for example, specifyingthat the first compositor 614 generate a display feed based on data fromonly the first device 630 for display on the first display 640 and thesecond compositor 616 generate a display feed based on data from device632 and device 634 for display on the second display 642. The differentconfigurations may be associated with different types of proceduresand/or different practitioners to, for example, support differentimaging sessions.

FIG. 7 illustrates a method 700 for configuring a medical imagingprocessing system, such as hub 400, according to some embodiments. Asdescribed further below, method 700 includes reconfiguring areconfigurable hardware processor, such as reconfigurable hardwareprocessor 408 of hub 400, according to predefined configuration dataassociated with a medical imaging session. The reconfigurable hardwareprocessor is configured to implement imaging data processing algorithmsdefined by the configuration data. Leveraging a reconfigurable hardwareprocessor to implement imaging processing algorithms allows for ahardware processor configuration that is tailored to processing ofimaging data according to the specified algorithms and allows fordifferent algorithms to be implemented for different imaging sessionshaving different imaging inputs and/or display requirements.Accordingly, the reconfigurable hardware processor can provideadvantages over a general purpose processor running software-basedalgorithms, which may not be able to provide the lower latency andhigher bandwidth that the reconfigurable processor can provide, whichare important in providing real time processing of video for displayduring a medical procedure.

At step 702, a reconfigurable hardware processor, such as reconfigurablehardware processor 408 of hub 400, is configured into a firstconfiguration for a first medical imaging session. The configuration ofthe hardware processor may be based on first configuration data storedin memory. The first configuration data may define one or more medicalimaging processing algorithms for implementation by the reconfigurablehardware processor. Once configured into the first configuration, thereconfigurable hardware processor implements the one or more medicalimaging processing algorithms as defined in the configuration data. Thereconfigurable hardware processor may be configured by loading one ormore hardware logic files from memory (which may be handled by a secondprocessor, such as primary processing unit 406 of hub 400) onto thereconfigurable hardware processor.

The first imaging session may include the performance of one or moremedical procedures, such as surgical procedures, on a patient. The firstimaging session may begin with a nurse or other user initializing themedical imaging system for a medical procedure or series of medicalprocedures on a patient. The first medical imaging session may becomplete when the medical procedure or all of a series of medicalprocedures on a patient are complete or may be complete when a first ofa series of procedures on a patient are complete. As an example of thislatter scenario, a first medical procedure on a patient, such as a firstsurgical procedure may complete, which completes a first imagingsession, and a second medical procedure, such as a different surgicalprocedure by the same or a different surgeon, may follow. The secondmedical procedure may include a second imaging session.

The reconfigurable hardware processor, in its first configuration, isconfigured to receive medical imaging data and to process at least aportion of the data using the first imaging processing algorithm. Insome embodiments, the first configuration may also include the abilityto process the received data using one or more additional processingalgorithms. The reconfigurable hardware processor in the firstconfiguration may have the ability to process multiple distinct sets ofimaging data (e.g., generated by different devices) using the firstimaging processing algorithm and/or additional imaging processingalgorithms. For example, the reconfigurable hardware processor in thefirst configuration may receive a first set of data generated by a firstconnected device and may process the first set of data using the firstmedical imaging processing algorithm and may also receive a second setof data generated by a second connected device and may process thesecond set of data using a second medical imaging processing algorithm.

In some embodiments, the reconfigurable hardware processor is configuredin response to an input indicative of the first medical imaging session.For example, a user, such as an operating room nurse, may provideinformation to the medical imaging processing system specifyingparameters associated with the imaging session, such as medicalprocedure type and/or practitioner identity. The one or more parametersmay be associated with the first configuration data and the system mayaccess the first configuration data and configure the reconfigurablehardware processor according to the specification of the firstconfiguration data. User input indicative of the first medical imagingsession may include a user selection of a profile. The profile may beassociated with one or more types of medical procedures and may definedata processing and display layout tailored to the one or more types ofmedical procedures. Types of medical procedure can include endoscopicmedical procedures, such as an enteroscopy, a colonoscopy, asigmoidoscopy, a rectoscopy, a rhinoscopy, a otoscopy, a cystoscopy, acolposcopy, a arthroscopy, a thoracoscopy, etc., and surgicalprocedures, such as a biopsy, a carotid endarterectomy, acholecystectomy, a coronary artery bypass, a skin graft, a hysterectomy,and a mastectomy.

The profile may be a practitioner profile that defines the types of datathat the practitioner wants to see in the layout that the practitionerprefers. The profile may be a default profile that includes a predefinedlayout and predefined data processing. The default profile may be basedon one or more parameters of the medical imaging system that aredetected by the medical imaging processing system, such as number andtypes of inputs to the processing system and number and types of displayoutputs from the processing system. In some embodiments, default profilemay be based on one or more parameters detected from received imagedata, such as a radius or diameter of an FOV, which may be associatedwith a type of imager (e.g., an endoscope size).

At step 704, the system receives first medical imaging data generatedduring the first medical imaging session. The first medical imaging datais received from one or more devices connected to one or more inputs ofthe system. For example, the first medical imaging data may be a seriesof video frames received from an imager, such as an endoscopic imager.The first medical imaging data may include data from multiple devicesconnected to the system, such as multiple video feeds from multipleimagers.

At step 706, enhanced first medical imaging data is generated at leastin part by processing the first medical imaging data using the firstmedical imaging processing algorithm implemented by the reconfigurablehardware processor in the first configuration. The reconfigurablehardware processor processes at least a portion of the first medicalimaging data using the first medical imaging processing algorithm andany other algorithms that the reconfigurable hardware processor isconfigured to implement, as defined by the first configuration data. Forexample, in the first configuration, the reconfigurable hardwareprocessor may implement a smoke detection algorithm that detectsportions of received images associated with smoke in the field of viewand enhances the received images to reduce the appearance of smoke.

Some or all of the first medical imaging data may be routed to thereconfigurable hardware processor by, for example, a primary processor,such as primary processing unit 406 of hub 400. The primary processormay receive the first medical imaging data and may route the data to thereconfigurable processor according to the first configuration data. Thefirst configuration data may specify that data received from a connecteddevice should be processed using at least the first medical imagingprocessing algorithm. In accordance with this requirement, the primaryprocessor may direct data received from the connected device to thereconfigurable hardware processor. In some embodiments, thereconfigurable processor receives the first medical imaging datadirectly from the input connection to the connected device—i.e., withoutthe data first being routed through one or more additional processingunits.

Processing of data by the reconfigurable hardware processor may be basedon processing by other processing units of the system. For example,processing of the first medical imaging data by the first medicalimaging processing algorithm may be based on information received from asecond processing unit. The second processing unit may analyze some orall of the first medical imaging data and the results of the analysismay be used by the reconfigurable hardware processor in theimplementation of the first medical imaging processing algorithm. Forexample, in the embodiment implementing a smoke detection algorithm inthe reconfigurable processor discussed above, an auxiliary processingunit, such as auxiliary processing unit 410 of hub 400, may receive someor all of the first imaging data to determine whether smoke is presentin the imaged field of view. Upon detecting smoke, the auxiliaryprocessing unit may notify the reconfigurable hardware processor (eitherdirectly or via another processing unit, such as primary processing unit406) and, in response, the reconfigurable hardware processor may beginprocessing the first imaging data to reduce the contribution of smoke inthe data.

At step 708, the enhanced first medical imaging data generated by thereconfigurable hardware processor is displayed for observation duringthe first medical imaging session. For example, the first medicalimaging session may include an endoscopic procedure that involves theuse of a cauterizing tool and the enhanced first medical imaging datamay be an enhancement of a video feed generated by an endoscopic camerain which the appearance of smoke generated by the cauterizing tool hasbeen reduced. This enhanced imaging may be displayed to the surgeon inreal time so that the surgeon can better visualize the surgical field.

In some embodiments, the enhanced first medical imaging data is receivedfrom the reconfigurable hardware processor by another processing unit,such as primary processing unit 406 of hub 400. The primary processormay generate one or more display feeds that include the enhanced firstmedical imaging data. The primary processor may generate the one or moredisplay feeds based at least in part on the first configuration data.For example, the primary processor may combine the enhanced firstmedical imaging data with additional information, such as additionalimaging received from another connected device, for display in differentparts of a display as defined by the first configuration data.

In some embodiments, the display feed includes the enhanced firstmedical imaging data combined with other data. For example, the displayfeed may include the enhanced first medical imaging data for display ina first portion of a connected display and may include additionalinformation for display in a second portion of the connected display. Insome embodiments, the system generates multiple display feeds havingdifferent display configurations for displaying the enhanced imagingdata and provides different display feeds to different displays.

FIG. 8A is a block diagram of a medical imaging processing system 800illustrating steps 702-708 of method 700, according to one embodiment.Reconfigurable hardware processor 804 is configured to process imagingdata received from a white light imager 810 via a first input port 808using a first imaging processing algorithm, such as a smoke detectionand removal algorithm. In response to a user input associated with afirst imaging session, the primary processor 802 accessed firstconfiguration data stored in the memory 812, and based on thespecifications in the first configuration data, the primary processor802 reconfigured reconfigurable hardware processor 804 by loading ahardware logic configuration file for a smoke reduction algorithm. Inaddition, the primary processor 802 loaded a smoke detection softwareprogram or module from the memory 812 to the auxiliary processor 806.The auxiliary processor 806 running the smoke detection software programor module can detect the presence of smoke in received imaging data andcan instruct the reconfigurable hardware processor 804 to process theimaging data to reduce the influence of smoke in the imaging data. Priorto detection of smoke by the auxiliary processor 806, the reconfigurablehardware processor may simply pass imaging data through for displaywithout first processing the data for smoke removal. The primaryprocessor may provide the auxiliary processor 806 with one or moreportions of the received imaging data (such as one or more frames), suchas on a periodic basis, for detecting smoke and triggering the smokeremoval processing of the reconfigurable hardware processor 804. Thesystem 800 outputs a display feed to display 816 that includes imagingdata received from the white light imager 810 that has been enhanced byremoving contributions from smoke when smoke is detected in the imagingdata.

Returning to method 700, at step 710, the reconfigurable hardwareprocessor is reconfigured into a second configuration for a secondmedical imaging session based on second configuration data stored in thememory. The second configuration implements at least one medical imagingprocessing algorithm that is not implemented in the first configuration.

In some embodiments, the reconfigurable hardware processor isreconfigured in response to an input indicative of the second medicalimaging session. For example, a user, such as an operating room nurse,may provide information to the medical imaging processing systemspecifying parameters associated with the second imaging session, suchas medical procedure type and/or practitioner identity. The one or moreparameters may be associated with the second configuration data and thesystem may access the second configuration data and configure thereconfigurable hardware processor according to the specifications of thesecond configuration data. User input indicative of the second medicalimaging session may include a user selection of a profile, as discussedabove. Depending on the profile selected, the second imaging session maybe associated with the same practitioner as the first imagingsession—for example, where the same practitioner is transitioning fromone type of medical procedure to another type of medical procedure thatmay require different display layouts, such as due to differentequipment connected to the medical imaging processing system ordifferent enhancement algorithms. The second imaging session may beassociated with the same type of medical procedure but differentpractitioners. For example, a first surgeon may perform a type ofsurgery (e.g., a cholecystectomy) on a first patient in the firstimaging session and a second surgeon may perform the same type ofsurgery (e.g., a cholecystectomy) on a second patient (e.g., later inthe day or on a following day).

The first imaging session may have completed (e.g., the surgery orprocedure associated with the first imaging session has completed) andthe imaging system may be set up for use in the subsequent secondimaging session. The second imaging session may involve one or moredifferent types of procedures and/or may include one or more differentusers for which different imaging processing may be beneficial.Accordingly, the second configuration implements one or more imageprocessing algorithms that were not implemented in the first imagingsession. The reconfigurable hardware processor is reconfigured toimplement the one or more image processing algorithms that are requiredfor the second imaging session, as defined by the second configurationdata.

In some embodiments, the second imaging session may be a second surgicalsession for which the imaging system is to be used. After completion ofthe first imaging session, the operating room may be set up for thesecond surgical session. The second surgical session may involve adifferent type of surgery, a different practitioner, a differentpatient, etc. During set up for the second surgical session, the imagingprocessing system may receive an input indicative of the second surgicalsession. The input can be, for example, a selection of a type ofsurgical procedure or a selection of a profile (e.g., a practitionerprofile) that is made via a user interface to the imaging processingsystem. Based on this selection, the system may automaticallyreconfigure the reconfigurable processor based on the configuration dataassociated with the second surgical session.

At step 712, the medical imaging processing system receives secondmedical imaging data generated during the second medical imagingsession. This second medical imaging data may be received from the sameconnected device or devices as the first medical imaging data or from adifferent connected device or devices.

At step 714, enhanced second medical imaging data is generated at leastin part by processing the second medical imaging data using the secondmedical imaging processing algorithm implemented in the secondconfiguration of the reconfigurable hardware processor. Thereconfigurable hardware processor processes at least a portion of thefirst medical imaging data using the second medical imaging processingalgorithm (which was not implemented in the first configuration) and anyother algorithms that the reconfigurable hardware processor wasconfigured to implement, which may or may not have been implemented inthe first configuration, as defined by the second configuration data.For example, in the second configuration, the reconfigurable hardwareprocessor may implement an algorithm that processes fluorescence images(e.g., video frames) to determine one or more features of blood flowthrough tissue, such as tissue perfusion, blood vessel location, bloodflow amounts or rates, dimensions of imaged tissue, or any combinationsthereof, and generates enhanced imaging modifying the fluorescenceimages according to the determined features (modifying coloring of theimages, overlaying data on the images, overlaying contouring on theimages, etc.).

As in the first configuration described above, some or all of the secondmedical imaging data may be routed to the reconfigurable hardwareprocessor by, for example, a primary processor, such as primaryprocessing unit 406 of hub 400. The primary processor may receive thesecond medical imaging data and may route the data to the reconfigurableprocessor according to the second configuration data. The secondconfiguration data may specify that data received from a connecteddevice should be processed using at least the second medical imagingprocessing algorithm. In accordance with this requirement, the primaryprocessor may direct data received from the connected device to thereconfigurable hardware processor. In some embodiments, thereconfigurable processor receives the first medical imaging datadirectly from the input, i.e., without the data first being routedthrough one or more additional processing units.

At step 716, the enhanced second medical imaging data generated by thereconfigurable hardware processor is displayed for observation duringthe second medical imaging session. Display of the enhanced secondmedical imaging data can assist a practitioner, such as a surgeon,during one or more procedures performed during the second medicalimaging session. By leveraging the low latency and high bandwidth of thereconfigurable processor, the enhanced second medical imaging data canbe displayed in real time.

In some embodiments, the enhanced second medical imaging data may betransmitted from the reconfigurable hardware processor to anotherprocessing unit, such as the primary processing unit 406 of FIG. 4,which may generate a display feed that includes the enhanced secondmedical imaging data. The display feed may be transmitted by the primaryprocessor to one or more connected displays. In some embodiments, thereconfigurable hardware processor may transmit the enhanced secondmedical imaging data directly to an output connection with one or moreconnected displays.

FIG. 8A is a block diagram of a medical imaging processing system 800illustrating steps 710-716 of method 700, according to one embodiment.Reconfigurable hardware processor 804 is reconfigured to process imagingdata received from the white light imager 810 and a fluorescent imager818 (these may be portions of the same imaging system and may bereceived on the same or different input ports) using a second imagingprocessing algorithm that analyzes the fluorescence imaging tocharacterize portions of tissue according to, for example, the health ofthe tissue, extent of blood flow in the tissue, or extent of perfusionin the tissue and overlays the characterization on the white lightimaging. In response to a user input associated with a second imagingsession (e.g., an input indicating a surgical session on a new patientor indicating a new procedure on the same patient as the first imagingsession), the primary processor 802 accessed second configuration datastored in the memory 812, and based on the specifications in the firstconfiguration data, the primary processor 802 reconfiguredreconfigurable hardware processor 804 by loading a hardware logicconfiguration file for the tissue characterization algorithm. Inaddition, the primary processor 802 loaded a reference marker softwareprogram or module from the memory 812 to the auxiliary processor 806.The auxiliary processor 806 running the reference marker program ormodule can determine locations of, for example, maximum and/or minimumperfusion in the fluorescence imaging data. Reference markers generatedby the auxiliary processor 806 are added to the overlay generated by thereconfigurable hardware processor (this may be done by thereconfigurable hardware processor 804, by the primary processor 802, orby a different processor of the system). The resulting enhanced imagingdata is output to display 816 for visualization during the secondimaging session.

According to some embodiments, the tissue characterization algorithmimplemented in the reconfigurable hardware processor may provideenhanced visual representations of tissue of a subject that may be moreaccurate in terms of data representation, and intuitive for cliniciansto use for their clinical decision making. The enhanced visualrepresentations of tissue generated may be applicable to various typesof tissue (e.g. a variety of wounds including chronic, acute, pressureulcers, cancerous tissue), and may provide a framework for automaticallyclassifying the tissue (e.g., wound tissue, cancerous tissue) and/orpredicting clinical outcomes (e.g., healing timeline for wound tissue,healing of cancerous tissue).

The tissue characterization algorithm may utilize machine learning ordeep learning. Machine learning-based methods and systems facilitatesolving problems that either do not have an algorithmic solution or asolution is too complex to find. Medical diagnosis and tissuecharacterization based on imaging of the tissue is a task particularlywell suited for machine learning algorithms due to complex nature ofphysiological processes taking place in the human body. Machine learningcan be used to discover medically-relevant features and patterns withinlarge datasets and help clinicians make medical diagnoses moreaccurately, more quickly and more consistently irrespective of theclinician's experience. In some embodiments, the tissue characterizationalgorithm includes identifying one or more attributes of the data thatare relevant to a clinical characterization of the tissue, andcategorizing the data into a plurality of clusters based on the one ormore attributes of the data such that the data in the same cluster aremore similar to each other than the data in different clusters, whereinthe clusters characterize the tissue. In some variations, the algorithmmay further include associating a respective cluster with each of aplurality of subregions in a time series of images such as for examplefluorescence images, and generating a subject spatial (cluster) mapbased on the associated clusters for the plurality of subregions in thesubject time series of fluorescence images. The algorithm may furtherinclude receiving a plurality of subject spatial maps and receivingmetadata associated with each subject spatial map, storing each subjectspatial map and its associated clinical data in a record of a database,and using the records of the database as input for a supervised machinelearning algorithm for generating a predictive model. The predictivemodel may be used for predicting clinical data associated with thesubject time series of fluorescence images of the subject.

FIGS. 9A and 9B illustrate graphical user interfaces for configuring amedical imaging processing system, such as hub 400, for a new imagingsession. The user interfaces may be provided, for example, on a tabletconnected to the system or on a touchscreen of the system. Userinterface 900 of FIG. 9A enables a user to configure an imagingprocessing system by selecting a specialty 902, a procedure 904, and/ora practitioner 906. Each selection may be associated with a differentconfiguration or combinations of selections may be associated withconfigurations. For example, each practitioner selection may beassociated with a different configuration, which has been previouslyspecified by the practitioner, whereas selection of both a specialty anda procedure may be required to select a configuration. Configurationsmay be stored locally in memory of the system or remotely in, forexample, a hospital information system, which is access, for example,via a network connection.

FIG. 9B illustrates a user interface 910 for defining data sources anddata source layouts. Two display layouts (912 and 914) are associatedwith the illustrated configuration. Each of the display layouts definesdata sources and data source size and location. The first display layout912 includes three different sources. As described above, sources candefine types of data displayed, which can be based on both the systemthat generates data and the types of processing on the data performed,for example, by the reconfigurable processor and/or other systemmodules. As such, different sources may be based on data from the sameimaging system or other devices. For example, source 1 can be stillimages from an input video stream (e.g., as selected via voice commandsfrom a practitioner) and source 3 can be the video stream. Userinterface 910 may enable a user to select, position, and size differentsources. For example, the user can select available sources from adrop-down list that specifies, for example, all sources that the systemis able to generate or all sources that the system is able to generategiven the inputs to the system. A user can drag source icons around thescreen to reposition the sources and can resize sources using, forexample, gestures, mouse inputs, keyboard inputs, or any other suitableinput.

Once a user completes selection of a configuration profile, the medicalimaging processing system may automatically configure itself accordingto the requirements defined in the selected configuration profile,according to the methods described above.

A system for collecting, enhancing, and displaying medical imaging data,such as system 100 of FIG. 1, may include one or more imaging systemsfor acquiring a time series of images of tissue (e.g., a time series offluorescence images, a time series of white light images, etc.). In someembodiments, an imaging system is a fluorescence imaging system. FIG. 10is a schematic example of a fluorescence imaging system 1010, accordingto one embodiment. The fluorescence imaging system 1010 comprises alight source 1012 to illuminate the tissue of the subject to inducefluorescence emission from a fluorescence imaging agent 1014 in thetissue of the subject (e.g., in blood, in urine, in lymph fluid, inspinal fluid or other body fluids or tissues), an image acquisitionassembly 1016 arranged for generating the time series and/or the subjecttime series of fluorescence images from the fluorescence emission, and aprocessor assembly 1018 arranged for processing the generated timeseries/subject time series of fluorescence images according to any ofthe variations of the methods described herein. The processor assembly1018 may include memory 1068 with instructions thereon, a processormodule 1062 arranged for executing the instructions on memory 1068 toprocess the time series and/or subject time series of fluorescenceimages, and a data storage module 1064 to store the unprocessed and/orprocessed time series and/or subject time series of fluorescence images.In some variations, the memory 1068 and data storage module 1064 may beembodied in the same storage medium, while in other variations thememory 1068 and the data storage module 1064 may be embodied indifferent storage mediums. The system 1010 may further include acommunication module 1066 for transmitting images and other data, suchas some or all of the time series/subject time series of fluorescenceimages or other input data, spatial maps, subject spatial maps, and/or atissue numerical value (quantifier), to an imaging data processing hub,such as imaging data processing hub 102 of FIG. 1, according to thesystems and methods discussed above.

In some variations, the light source 1012 includes, for example, anillumination module 1020. Illumination module 1020 may include afluorescence excitation source arranged for generating an excitationlight having a suitable intensity and a suitable wavelength for excitingthe fluorescence imaging agent 1014. As shown in FIG. 11, theillumination module 1020 may comprise a laser diode 1022 (e.g., whichmay comprise, for example, one or more fiber-coupled diode lasers)arranged for providing an excitation light to excite the fluorescenceimaging agent (not shown) in tissue of the subject. Examples of othersources of the excitation light which may be used in various embodimentsinclude one or more LEDs, arc lamps, or other illuminant technologies ofsufficient intensity and appropriate wavelength to excite thefluorescence imaging agent in the tissue. For example, excitation of thefluorescence imaging agent in blood, wherein the fluorescence imagingagent is a fluorescence dye with near infra-red excitation and emissioncharacteristics, may be performed using one or more 793 nm,conduction-cooled, single bar, fiber-coupled laser diode modules fromDILAS Diode Laser Co, Germany.

In some variations, the light output from the light source 1012 may beprojected through one or more optical elements to shape and guide theoutput being used to illuminate the tissue area of interest. The opticalelements may include one or more lenses, light guides, and/ordiffractive elements so as to ensure a flat field over substantially theentire field of view of the image acquisition assembly 1016. Thefluorescence excitation source may be selected to emit at a wavelengthclose to the absorption maximum of the fluorescence imaging agent 1014(e.g., indocyanine green (ICG), etc.). For example, as shown in FIG. 11,the output 1024 from the laser diode 1022 may be passed through one ormore focusing lenses 1026, and then through a homogenizing light pipe1028 such as, for example, light pipes commonly available from NewportCorporation, USA. Finally, the light may be passed through an opticaldiffractive element 1032 (i.e., one or more optical diffusers) such as,for example, ground glass diffractive elements also available fromNewport Corporation, USA. Power to the laser diode 1022 may be providedby, for example, a high-current laser driver such as those availablefrom Lumina Power Inc. USA. The laser may optionally be operated in apulsed mode during the image acquisition process. An optical sensor suchas a solid state photodiode 1030 may be incorporated into theillumination module 1020 and may sample the illumination intensityproduced by the illumination module 1020 via scattered or diffusereflections from the various optical elements. In some variations,additional illumination sources may be used to provide guidance whenaligning and positioning the module over the area of interest.

Referring again to FIG. 10, in some variations, the image acquisitionassembly 1016 may be a component of a fluorescence imaging system 1010configured to acquire the time series and/or subject time series offluorescence images from the fluorescence emission from the fluorescenceimaging agent 1014. The image acquisition assembly 1016 may include acamera module 1040. As shown in FIG. 12, the camera module 1040 mayacquire images of the fluorescence emission 1042 from the fluorescenceimaging agent in the tissue by using a system of imaging optics (e.g.,1046 a, 1046 b, 1048 and 1050) to collect and focus the fluorescenceemission onto an image sensor assembly 1044. The image sensor assembly1044 may comprise at least one 2D solid state image sensor. The solidstate image sensor may be a charge coupled device (CCD), a CMOS sensor,a CID or similar 2D sensor technology. The charge that results from theoptical signal transduced by the image sensor assembly 1044 is convertedto an electrical video signal, which includes both digital and analogvideo signals, by the appropriate read-out and amplification electronicsin the camera module 1040.

According to an exemplary variation of a fluorescent imaging system, thelight source may provide an excitation wavelength of about 800 nm+/−10nm, and the image acquisition assembly uses emission wavelengths of >820nm with NIR-compatible optics for, for example, ICG fluorescenceimaging. In an exemplary embodiment, the NIR-compatible optics mayinclude a CCD monochrome image sensor having a GigE standard interfaceand a lens that is compatible with the sensor with respect to opticalformat and mount format (e.g., C/CS mount).

In some variations, the processor module 1062 comprises any computer orcomputing means such as, for example, a tablet, laptop, desktop,networked computer, or dedicated standalone microprocessor. Forinstance, the processor module 1062 may include one or more centralprocessing units (CPU). In an exemplary embodiment, the processor module1062 is a quad-core, 2.5 GHz processor with four CPUs where each CPU isa microprocessor such as a 64-bit microprocessor (e.g., marketed asINTEL Core i3, i5, or i7, or in the AMD Core FX series). However, inother embodiments, the processor module 1062 may be any suitableprocessor with any suitable number of CPUs and/or other suitable clockspeed.

Inputs for the processor module 1062 may be taken, for example, from theimage sensor 1044 of the camera module 1040 shown in FIG. 12, from thesolid state photodiode 1030 in the illumination module 1020 in FIG. 11,and/or from any external control hardware such as a footswitch orremote-control. Output is provided to the laser diode driver and opticalalignment aids. As shown in FIG. 10, in some variations, the processorassembly 1018 may have a data storage module 1064 with the capability tosave the time series/subject time series of images, or datarepresentative thereof, or other input data to a tangible non-transitorycomputer readable medium such as, for example, internal memory (e.g. ahard disk or flash memory), so as to enable recording and processing ofacquired data. In some variations, the processor module 1062 may have aninternal clock to enable control of the various elements and ensurecorrect timing of illumination and sensor shutters. In some variations,the processor module 1062 may also provide user input and graphicaldisplay of outputs. The fluorescence imaging system may optionally beconfigured with a communication unit 1066, such as a wired or wirelessnetwork connection or video output connection for transmitting the timeseries of fluorescence images as they are being acquired or played backafter recording. The communication unit 1066 may additionally oralternatively transmit processed data, such as a spatial map, a subjectspatial map, and/or tissue numerical value.

In operation of the exemplary system described in FIGS. 10-12, thesubject is positioned relative to fluorescence imaging system 1010 suchthat an area of interest (e.g., target tissue region) is located beneaththe light source 1012 and the image acquisition assembly 1016 such thatthe illumination module 1020 of light source 1012 produces asubstantially uniform field of illumination across substantially theentire area of interest. In some variations, prior to the administrationof the fluorescence imaging agent 1014 to the subject, an image may beacquired of the area of interest for the purposes of backgrounddeduction. To acquire fluorescence images/subject fluorescence images,the operator of the fluorescence imaging system 1010 may initiate theacquisition of the time series/subject time series of fluorescenceimages by depressing a remote switch or foot-control, or via a keyboard(not shown) connected to the processor assembly 1018. As a result, thelight source 1012 is turned on and the processor assembly 1018 beginsrecording the fluorescence image data/subject fluorescence image dataprovided by the image acquisition assembly 1016. When operating in thepulsed mode of the embodiment, the image sensor 1044 in the cameramodule 1040 is synchronized to collect fluorescence emission followingthe laser pulse produced by the diode laser 822 in the illuminationmodule 1020. In this way, maximum fluorescence emission intensity isrecorded, and signal-to-noise ratio is optimized. In this embodiment,the fluorescence imaging agent 1014 is administered to the subject anddelivered to the area of interest via arterial flow. Acquisition of thetime series/subject time series of fluorescence images is initiated, forexample, shortly after administration of the fluorescence imaging agent1014, and the time series of fluorescence images from substantially theentire area of interest is acquired throughout the ingress of thefluorescence imaging agent 1014. The fluorescence emission from theregion of interest is collected by the collection optics of the cameramodule 1040. Residual ambient and reflected excitation light isattenuated by subsequent optical elements (e.g., optical element 1050 inFIG. 12 which may be a filter) in the camera module 1040 so that thefluorescence emission can be acquired by the image sensor assembly 1044with minimal interference by light from other sources.

In some variations, following the acquisition or generation of the timeseries/subject time series of fluorescence images, the processorassembly 1018 (e.g., processor module 1062 or other processor) may thenbe initiated to execute instructions stored on memory 1068 and processthe imaging data before transmission to the imaging data processingsystem (e.g., hub 102 of system 100). The system 1010 may transmit, viaconnection 1066, the spatial map/subject spatial map and/or any clinicalcorrelations or diagnosis derived therefrom or both for display to theuser in a composite display feed as, for example, a grayscale or falsecolor image, and/or stored for subsequent use.

FIG. 13 shows an endoscopic surgical cart embodiment of system 100 ofFIG. 1. Cart 10 may be used, for example, in an operating room forendoscopic imaging and display during an endoscopic procedure. Cart 10includes an imaging system, such as fluorescence imaging system 1010 ofFIG. 10. The imaging system includes a scope assembly 11 which may beutilized in endoscopic procedures. The scope assembly 11 incorporates anendoscope or scope 12 which is coupled to a camera head 16 by a coupler13 located at the distal end of the camera head 16. Light is provided tothe scope by a light source 14 via a light guide 26, such as a fiberoptic cable. The camera head 16 is coupled to a camera control unit(CCU) 18 by an electrical cable 15. The CCU 18 is preferably connectedto, and communicates with, the light source 14. Operation of the camera16 is controlled, in part, by the CCU 18. The cable 15 conveys videoimage data from the camera head 16 to the CCU 18 and conveys variouscontrol signals bi-directionally between the camera head 16 and the CCU18. In one embodiment, the image data output by the camera head 16 isdigital.

A control or switch arrangement 17 is provided on the camera head 16 andallows a user to manually control various functions of the cart 10.Voice commands may be input into a microphone 25 mounted on a headset 27worn by a surgeon and coupled to a voice-control unit 23. Cart 10 mayinclude a hand-held control device 21, such as a tablet with a touchscreen user interface or a PDA, that may be coupled to the cart 10 as afurther control interface. The cart 10 also includes an imaging dataprocessing hub 31, such as hub 102 of FIG. 1 or hub 400 of FIG. 4, whichis coupled to the imaging system via one or more cable connections forreceiving images and/or video from the imaging system, processing theimages and/or video, and generating display feeds for display on display20 according to the methods described herein. The imaging dataprocessing hub may receive user input via the voice control unit and/orthrough the hand-held control device.

Cart 10 may include one or more additional devices 33, such as animaging recording device or a surgical tool control device, which may becoupled to the imaging data processing hub. The imaging data processinghub 31 may receive information from the one or more additional devices33, such as device warnings, device status, and device settings. In someembodiments, the additional device 33 is a video recorder, and theimaging data processing hub 31 may transmit one or more display feeds tothe video recorder for recording.

A tangible non-transitory computer readable medium havingcomputer-executable (readable) program code embedded thereon may provideinstructions for causing one or more processors to, when executing theinstructions, perform one or more of the methods described herein.Program code can be written in any appropriate programming language anddelivered to the processor in many forms, including, for example, butnot limited to information permanently stored on non-writeable storagemedia (e.g., read-only memory devices such as ROMs, CD-ROM disks, etc.),information alterably stored on writeable storage media (e.g., harddrives or the like), information conveyed to the processor throughcommunication media, such as a local area network, a public network suchas the Internet, or any type of media suitable for storing electronicinstruction. When carrying computer readable instructions that implementthe various embodiments of the methods described herein, such computerreadable media represent examples of various embodiments. In variousembodiments, the tangible non-transitory computer readable mediumcomprises all computer-readable media, and the present invention scopeis limited to computer readable media wherein the media is both tangibleand non-transitory.

A kit may include any part of the systems described herein and thefluorescence imaging agent such as, for example, a fluorescence dye suchas ICG or any suitable fluorescence imaging agent. In further aspects, akit may include a tangible non-transitory computer readable mediumhaving computer-executable (readable) program code embedded thereon thatmay provide instructions for causing one or more processors, whenexecuting the instructions, to perform one or more of the methods forcharacterizing tissue and/or predicting clinical data described herein.The kit may include instructions for use of at least some of itscomponents (e.g., for using the fluorescence imaging agent, forinstalling the computer-executable (readable) program code withinstructions embedded thereon, etc.). In yet further aspects, there isprovided a fluorescence imaging agent such as, for example, afluorescence dye for use in in the methods and systems described herein.In further variations, a kit may include any part of or the entiresystem described herein and a fluorescence agent such as, for example, afluorescence dye such as ICG, or any other suitable fluorescence agent,or a combination of fluorescence agents.

Example Imaging Agents for Use in Generating Imaging Data

According to some embodiments, in fluorescence medical imagingapplications, the imaging agent is a fluorescence imaging agent such as,for example, ICG dye. ICG, when administered to the subject, binds withblood proteins and circulates with the blood in the tissue. Thefluorescence imaging agent (e.g., ICG) may be administered to thesubject as a bolus injection (e.g., into a vein or an artery) in aconcentration suitable for imaging such that the bolus circulates in thevasculature and traverses the microvasculature. In other embodiments inwhich multiple fluorescence imaging agents are used, such agents may beadministered simultaneously, e.g. in a single bolus, or sequentially inseparate boluses. In some embodiments, the fluorescence imaging agentmay be administered by a catheter. In certain embodiments, thefluorescence imaging agent may be administered less than an hour inadvance of performing the measurement of signal intensity arising fromthe fluorescence imaging agent. For example, the fluorescence imagingagent may be administered to the subject less than 30 minutes in advanceof the measurement. In yet other embodiments, the fluorescence imagingagent may be administered at least 30 seconds in advance of performingthe measurement. In still other embodiments, the fluorescence imagingagent may be administered contemporaneously with performing themeasurement.

According to some embodiments, the fluorescence imaging agent may beadministered in various concentrations to achieve a desired circulatingconcentration in the blood. For example, in embodiments where thefluorescence imaging agent is ICG, it may be administered at aconcentration of about 2.5 mg/mL to achieve a circulating concentrationof about 5 μM to about 10 μM in blood. In various embodiments, the upperconcentration limit for the administration of the fluorescence imagingagent is the concentration at which the fluorescence imaging agentbecomes clinically toxic in circulating blood, and the lowerconcentration limit is the instrumental limit for acquiring the signalintensity data arising from the fluorescence imaging agent circulatingwith blood to detect the fluorescence imaging agent. In various otherembodiments, the upper concentration limit for the administration of thefluorescence imaging agent is the concentration at which thefluorescence imaging agent becomes self-quenching. For example, thecirculating concentration of ICG may range from about 2 μM to about 10mM. Thus, in one aspect, the method comprises the step of administrationof the imaging agent (e.g., a fluorescence imaging agent) to the subjectand acquisition of the signal intensity data (e.g., video) prior toprocessing the signal intensity data according to the variousembodiments. In another aspect, the method excludes any step ofadministering the imaging agent to the subject.

According to some embodiments, a suitable fluorescence imaging agent foruse in fluorescence imaging applications to generate fluorescence imagedata is an imaging agent which can circulate with the blood (e.g., afluorescence dye which can circulate with, for example, a component ofthe blood such as lipoproteins or serum plasma in the blood) and transitvasculature of the tissue (i.e., large vessels and microvasculature),and from which a signal intensity arises when the imaging agent isexposed to appropriate light energy (e.g., excitation light energy, orabsorption light energy). In various embodiments, the fluorescenceimaging agent comprises a fluorescence dye, an analogue thereof, aderivative thereof, or a combination of these. A fluorescence dyeincludes any non-toxic fluorescence dye. In certain embodiments, thefluorescence dye optimally emits fluorescence in the near-infraredspectrum. In certain embodiments, the fluorescence dye is or comprises atricarbocyanine dye. In certain embodiments, the fluorescence dye is orcomprises ICG, methylene blue, or a combination thereof. In otherembodiments, the fluorescence dye is or comprises fluoresceinisothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin,o-phthaldehyde, fluorescamine, rose Bengal, trypan blue, fluoro-gold, ora combination thereof, excitable using excitation light wavelengthsappropriate to each dye. In some embodiments, an analogue or aderivative of the fluorescence dye may be used. For example, afluorescence dye analog or a derivative includes a fluorescence dye thathas been chemically modified, but still retains its ability to fluorescewhen exposed to light energy of an appropriate wavelength.

In various embodiments, the fluorescence imaging agent may be providedas a lyophilized powder, solid, or liquid. In certain embodiments, thefluorescence imaging agent may be provided in a vial (e.g., a sterilevial), which may permit reconstitution to a suitable concentration byadministering a sterile fluid with a sterile syringe. Reconstitution maybe performed using any appropriate carrier or diluent. For example, thefluorescence imaging agent may be reconstituted with an aqueous diluentimmediately before administration. In various embodiments, any diluentor carrier which will maintain the fluorescence imaging agent insolution may be used. As an example, ICG may be reconstituted withwater. In some embodiments, once the fluorescence imaging agent isreconstituted, it may be mixed with additional diluents and carriers. Insome embodiments, the fluorescence imaging agent may be conjugated toanother molecule, such as a protein, a peptide, an amino acid, asynthetic polymer, or a sugar, for example to enhance solubility,stability, imaging properties, or a combination thereof. Additionalbuffering agents may optionally be added including Tris, HCl, NaOH,phosphate buffer, and/or HEPES.

A person of skill in the art will appreciate that, although afluorescence imaging agent was described above in detail, other imagingagents may be used in connection with the systems, methods, andtechniques described herein, depending on the optical imaging modality.Such fluorescence agents may be administered into body fluid (e.g.,lymph fluid, spinal fluid) or body tissue.

In some variations, the fluorescence imaging agent used in combinationwith the methods, systems and kits described herein may be used forblood flow imaging, tissue perfusion imaging, lymphatic imaging, or acombination thereof, which may performed during an invasive surgicalprocedure, a minimally invasive surgical procedure, a non-invasivesurgical procedure, or a combination thereof. Examples of invasivesurgical procedure which may involve blood flow and tissue perfusioninclude a cardiac-related surgical procedure (e.g., CABG on pump or offpump) or a reconstructive surgical procedure. An example of anon-invasive or minimally invasive procedure includes wound (e.g.,chronic wound such as for example pressure ulcers) treatment and/ormanagement. In this regard, for example, a change in the wound overtime, such as a change in wound dimensions (e.g., diameter, area), or achange in tissue perfusion in the wound and/or around the peri-wound,may be tracked over time with the application of the methods andsystems. Examples of lymphatic imaging include identification of one ormore lymph nodes, lymph node drainage, lymphatic mapping, or acombination thereof. In some variations such lymphatic imaging mayrelate to the female reproductive system (e.g., uterus, cervix, vulva).

In variations relating to cardiac applications, the imaging agent(s)(e.g., ICG alone or in combination with another imaging agent) may beinjected intravenously through, for example, the central venous line,bypass pump and/or cardioplegia line to flow and/or perfuse the coronaryvasculature, microvasculature and/or grafts. ICG may be administered asa dilute ICG/blood/saline solution down the grafted vessel such that thefinal concentration of ICG in the coronary artery is approximately thesame or lower as would result from injection of about 2.5 mg (i.e., 1 mlof 2.5 mg/ml) into the central line or the bypass pump. The ICG may beprepared by dissolving, for example, 25 mg of the solid in 10 ml sterileaqueous solvent, which may be provided with the ICG by the manufacturer.One milliliter of the ICG solution may be mixed with 500 ml of sterilesaline (e.g., by injecting 1 ml of ICG into a 500 ml bag of saline).Thirty milliliters of the dilute ICG/saline solution may be added to 10ml of the subject's blood, which may be obtained in an aseptic mannerfrom the central arterial line or the bypass pump. ICG in blood binds toplasma proteins and facilitates preventing leakage out of the bloodvessels. Mixing of ICG with blood may be performed using standardsterile techniques within the sterile surgical field. Ten ml of theICG/saline/blood mixture may be administered for each graft. Rather thanadministering ICG by injection through the wall of the graft using aneedle, ICG may be administered by means of a syringe attached to the(open) proximal end of the graft. When the graft is harvested surgeonsroutinely attach an adaptor to the proximal end of the graft so thatthey can attach a saline filled syringe, seal off the distal end of thegraft and inject saline down the graft, pressurizing the graft and thusassessing the integrity of the conduit (with respect to leaks, sidebranches etc.) prior to performing the first anastomosis. In othervariations, the methods, dosages or a combination thereof as describedherein in connection with cardiac imaging may be used in any vascularand/or tissue perfusion imaging applications.

Lymphatic mapping is an important part of effective surgical staging forcancers that spread through the lymphatic system (e.g., breast, gastric,gynecological cancers). Excision of multiple nodes from a particularnode basin can lead to serious complications, including acute or chroniclymphedema, paresthesia, and/or seroma formation, when in fact, if thesentinel node is negative for metastasis, the surrounding nodes willmost likely also be negative. Identification of the tumor draining lymphnodes (LN) has become an important step for staging cancers that spreadthrough the lymphatic system in breast cancer surgery for example. LNmapping involves the use of dyes and/or radiotracers to identify the LNseither for biopsy or resection and subsequent pathological assessmentfor metastasis. The goal of lymphadenectomy at the time of surgicalstaging is to identify and remove the LNs that are at high risk forlocal spread of the cancer. Sentinel lymph node (SLN) mapping hasemerged as an effective surgical strategy in the treatment of breastcancer. It is generally based on the concept that metastasis (spread ofcancer to the axillary LNs), if present, should be located in the SLN,which is defined in the art as the first LN or group of nodes to whichcancer cells are most likely to spread from a primary tumor. If the SLNis negative for metastasis, then the surrounding secondary and tertiaryLN should also be negative. The primary benefit of SLN mapping is toreduce the number of subjects who receive traditional partial orcomplete lymphadenectomy and thus reduce the number of subjects whosuffer from the associated morbidities such as lymphedema andlymphocysts.

The current standard of care for SLN mapping involves injection of atracer that identifies the lymphatic drainage pathway from the primarytumor. The tracers used may be radioisotopes (e.g. Technetium-99 orTc-99m) for intraoperative localization with a gamma probe. Theradioactive tracer technique (known as scintigraphy) is limited tohospitals with access to radioisotopes require involvement of a nuclearphysician and does not provide real-time visual guidance. A colored dye,isosulfan blue, has also been used, however this dye cannot be seenthrough skin and fatty tissue. In addition, blue staining results intattooing of the breast lasting several months, skin necrosis can occurwith subdermal injections, and allergic reactions with rare anaphylaxishave also been reported. Severe anaphylactic reactions have occurredafter injection of isosulfan blue (approximately 2% of patients).Manifestations include respiratory distress, shock, angioedema,urticarial and pruritus. Reactions are more likely to occur in subjectswith a history of bronchial asthma, or subjects with allergies or drugreactions to triphenylmethane dyes. Isosulfan blue is known to interferewith measurements of oxygen saturation by pulse oximetry andmethemoglobin by gas analyzer. The use of isosulfan blue may result intransient or long-term (tattooing) blue coloration.

In contrast, fluorescence imaging in accordance with the variousembodiments for use in SLN visualization, mapping, facilitates directreal-time visual identification of a LN and/or the afferent lymphaticchannel intraoperatively, facilitates high-resolution optical guidancein real-time through skin and fatty tissue, visualization of blood flow,tissue perfusion or a combination thereof.

In some variations, visualization, classification or both of lymph nodesduring fluorescence imaging may be based on imaging of one or moreimaging agents, which may be further based on visualization and/orclassification with a gamma probe (e.g., Technetium Tc-99m is a clear,colorless aqueous solution and is typically injected into theperiareolar area as per standard care), another conventionally usedcolored imaging agent (isosulfan blue), and/or other assessment such as,for example, histology. The breast of a subject may be injected, forexample, twice with about 1% isosulfan blue (for comparison purposes)and twice with an ICG solution having a concentration of about 2.5mg/ml. The injection of isosulfan blue may precede the injection of ICGor vice versa. For example, using a TB syringe and a 30 G needle, thesubject under anesthesia may be injected with 0.4 ml (0.2 ml at eachsite) of isosulfan blue in the periareolar area of the breast. For theright breast, the subject may be injected at 12 and 9 o'clock positionsand for the left breast at 12 and 3 o'clock positions. The total dose ofintradermal injection of isosulfan blue into each breast may be about4.0 mg (0.4 ml of 1% solution: 10 mg/ml). In another exemplaryvariation, the subject may receive an ICG injection first followed byisosulfan blue (for comparison). One 25 mg vial of ICG may bereconstituted with 10 ml sterile water for injection to yield a 2.5mg/ml solution immediately prior to ICG administration. Using a TBsyringe and a 30G needle, for example, the subject may be injected withabout 0.1 ml of ICG (0.05 ml at each site) in the periareolar area ofthe breast (for the right breast, the injection may be performed at 12and 9 o'clock positions and for the left breast at 12 and 3 o'clockpositions). The total dose of intradermal injection of ICG into eachbreast may be about 0.25 mg (0.1 ml of 2.5 mg/ml solution) per breast.ICG may be injected, for example, at a rate of 5 to 10 seconds perinjection. When ICG is injected intradermally, the protein bindingproperties of ICG cause it to be rapidly taken up by the lymph and movedthrough the conducting vessels to the LN. In some variations, the ICGmay be provided in the form of a sterile lyophilized powder containing25 mg ICG with no more than 5% sodium iodide. The ICG may be packagedwith aqueous solvent consisting of sterile water for injection, which isused to reconstitute the ICG. In some variations the ICG dose (mg) inbreast cancer sentinel lymphatic mapping may range from about 0.5 mg toabout 10 mg depending on the route of administration. In somevariations, the ICG does may be about 0.6 mg to about 0.75 mg, about0.75 mg to about 5 mg, about 5 mg to about 10 mg. The route ofadministration may be for example subdermal, intradermal (e.g., into theperiareolar region), subareolar, skin overlaying the tumor, intradermalin the areola closest to tumor, subdermal into areola, intradermal abovethe tumor, periareolar over the whole breast, or a combination thereof.The NIR fluorescent positive LNs (e.g., using ICG) may be represented asa black and white NIR fluorescence image(s) for example and/or as a fullor partial color (white light) image, full or partial desaturated whitelight image, an enhanced colored image, an overlay (e.g., fluorescencewith any other image), a composite image (e.g., fluorescenceincorporated into another image) which may have various colors, variouslevels of desaturation or various ranges of a color tohighlight/visualize certain features of interest. Processing of theimages may be further performed for further visualization and/or otheranalysis (e.g., quantification). The lymph nodes and lymphatic vesselsmay be visualized (e.g., intraoperatively, in real time) usingfluorescence imaging systems and methods according to the variousembodiments for ICG and SLNs alone or in combination with a gamma probe(Tc-99m) according to American Society of Breast Surgeons (ASBrS)practice guidelines for SLN biopsy in breast cancer patients.Fluorescence imaging for LNs may begin from the site of injection bytracing the lymphatic channels leading to the LNs in the axilla. Oncethe visual images of LNs are identified, LN mapping and identificationof LNs may be done through incised skin, LN mapping may be performeduntil ICG visualized nodes are identified. For comparison, mapping withisosulfan blue may be performed until ‘blue’ nodes are identified. LNsidentified with ICG alone or in combination with another imagingtechnique (e.g., isosulfan blue, and/or Tc-99m) may be labeled to beexcised. Subject may have various stages of breast cancer (e.g., IA, IB,IIA).

In some variations, such as for example, in gynecological cancers (e.g.,uterine, endometrial, vulvar and cervical malignancies), ICG may beadministered interstitially for the visualization of lymph nodes,lymphatic channels, or a combination thereof. When injectedinterstitially, the protein binding properties of ICG cause it to berapidly taken up by the lymph and moved through the conducting vesselsto the SLN. ICG may be provided for injection in the form of a sterilelyophilized powder containing 25 mg ICG (e.g., 25 mg/vial) with no morethan 5.0% sodium iodide. ICG may be then reconstituted with commerciallyavailable water (sterile) for injection prior to use. According to anembodiment, a vial containing 25 mg ICG may be reconstituted in 20 ml ofwater for injection, resulting in a 1.25 mg/ml solution. A total of 4 mlof this 1.25 mg/ml solution is to be injected into a subject (4×1 mlinjections) for a total dose of ICG of 5 mg per subject. The cervix mayalso be injected four (4) times with a 1 ml solution of 1% isosulfanblue 10 mg/ml (for comparison purposes) for a total dose of 40 mg. Theinjection may be performed while the subject is under anesthesia in theoperating room. In some variations the ICG dose (mg) in gynecologicalcancer sentinel lymph node detection and/or mapping may range from about0.1 mg to about 5 mg depending on the route of administration. In somevariations, the ICG does may be about 0.1 mg to about 0.75 mg, about0.75 mg to about 1.5 mg, about 1.5 mg to about 2.5 mg, about 2.5 mg toabout 5 mg. The route of administration may be for example cervicalinjection, vulva peritumoral injection, hysteroscopic endometrialinjection, or a combination thereof. In order to minimize the spillageof isosulfan blue or ICG interfering with the mapping procedure when LNsare to be excised, mapping may be performed on a hemi-pelvis, andmapping with both isosulfan blue and ICG may be performed prior to theexcision of any LNs. LN mapping for Clinical Stage I endometrial cancermay be performed according to the NCCN Guidelines for Uterine Neoplasms,SLN Algorithm for Surgical Staging of Endometrial Cancer; and SLNmapping for Clinical Stage I cervical cancer may be performed accordingto the NCCN Guidelines for Cervical Neoplasms, Surgical/SLN MappingAlgorithm for Early-Stage Cervical Cancer. Identification of LNs maythus be based on ICG fluorescence imaging alone or in combination orco-administration with for a colorimetric dye (isosulfan blue) and/orradiotracer.

Visualization of lymph nodes may be qualitative and/or quantitative.Such visualization may comprise, for example, lymph node detection,detection rate, anatomic distribution of lymph nodes. Visualization oflymph nodes according to the various embodiments may be used alone or incombination with other variables (e.g., vital signs, height, weight,demographics, surgical predictive factors, relevant medical history andunderlying conditions, histological visualization and/or assessment,Tc-99m visualization and/or assessment, concomitant medications).Follow-up visits may occur on the date of discharge, and subsequentdates (e.g., one month).

Lymph fluid comprises high levels of protein, thus ICG can bind toendogenous proteins when entering the lymphatic system. Fluorescenceimaging (e.g., ICG imaging) for lymphatic mapping when used inaccordance with the methods and systems described herein offers thefollowing example advantages: high-signal to background ratio (or tumorto background ratio) as NIR does not generate significantautofluorescence, real-time visualization feature for lymphatic mapping,tissue definition (i.e., structural visualization), rapid excretion andelimination after entering the vascular system, and avoidance ofnon-ionizing radiation. Furthermore, NIR imaging has superior tissuepenetration (approximately 5 to 10 millimeters of tissue) to that ofvisible light (1 to 3 mm of tissue). The use of ICG for example alsofacilitates visualization through the peritoneum overlying thepara-aortic nodes. Although tissue fluorescence can be observed with NIRlight for extended periods, it cannot be seen with visible light andconsequently does not impact pathologic evaluation or processing of theLN. Also, florescence is easier to detect intra-operatively than bluestaining (isosulfan blue) of lymph nodes. In other variations, themethods, dosages or a combination thereof as described herein inconnection with lymphatic imaging may be used in any vascular and/ortissue perfusion imaging applications.

Tissue perfusion relates to the microcirculatory flow of blood per unittissue volume in which oxygen and nutrients are provided to and waste isremoved from the capillary bed of the tissue being perfused. Tissueperfusion is a phenomenon related to but also distinct from blood flowin vessels. Quantified blood flow through blood vessels may be expressedin terms that define flow (i.e., volume/time), or that define speed(i.e., distance/time). Tissue blood perfusion defines movement of bloodthrough micro-vasculature, such as arterioles, capillaries, or venules,within a tissue volume. Quantified tissue blood perfusion may beexpressed in terms of blood flow through tissue volume, namely, that ofblood volume/time/tissue volume (or tissue mass). Perfusion isassociated with nutritive blood vessels (e.g., micro-vessels known ascapillaries) that comprise the vessels associated with exchange ofmetabolites between blood and tissue, rather than larger-diameternon-nutritive vessels. In some embodiments, quantification of a targettissue may include calculating or determining a parameter or an amountrelated to the target tissue, such as a rate, size volume, time,distance/time, and/or volume/time, and/or an amount of change as itrelates to any one or more of the preceding parameters or amounts.However, compared to blood movement through the larger diameter bloodvessels, blood movement through individual capillaries can be highlyerratic, principally due to vasomotion, wherein spontaneous oscillationin blood vessel tone manifests as pulsation in erythrocyte movement. Insome embodiments, blood flow and tissue perfusion imaging describedherein in connection with the systems and methods may be used to imagetumor tissue and differentiate such tissue from other tissue.

The foregoing description, for the purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the techniques and their practical applications. Othersskilled in the art are thereby enabled to best utilize the techniquesand various embodiments with various modifications as are suited to theparticular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying figures, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims. Finally, the entire disclosure of the patents andpublications referred to in this application are hereby incorporatedherein by reference.

1. A method of configuring a medical imaging processing system, the method comprising: configuring a reconfigurable hardware processor of the medical imaging processing system into a first configuration for a first medical imaging session based on first configuration data stored in a memory, wherein the first configuration implements at least a first medical imaging processing algorithm; receiving first medical imaging data generated during the first medical imaging session; generating enhanced first medical imaging data at least in part by processing the first medical imaging data using the first medical imaging processing algorithm implemented in the first configuration; displaying the enhanced first medical imaging data for observation during the first medical imaging session; reconfiguring the reconfigurable hardware processor into a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the second configuration implements at least a second medical imaging processing algorithm that is not implemented in the first configuration; receiving second medical imaging data generated during the second medical imaging session; generating enhanced second medical imaging data at least in part by processing the second medical imaging data using the second medical imaging processing algorithm implemented in the second configuration; and displaying the enhanced second medical imaging data for observation during the second medical imaging session on a display.
 2. The method of claim 1, comprising receiving an input indicative of the second medical imaging session and, in response to receiving the input, automatically reconfiguring the reconfigurable hardware processor into the second configuration.
 3. The method of claim 2, wherein the input comprises a selection of a type of medical procedure.
 4. The method of claim 2, wherein the input comprises a selection of a user profile.
 5. The method of claim 2, wherein the input comprises selection of a default configuration profile.
 6. The method of claim 5, wherein the default configuration profile is based one or more connections to the medical imaging processing system from one or more external devices.
 7. The method of claim 6, wherein the default configuration profile is based on a field of view of a connected external device.
 8. The method of claim 1, wherein the first configuration is associated with a first type of medical procedure and the second configuration is associated with a second type of medical procedure.
 9. The method of claim 8, wherein the first medical imaging session includes performance of the first type of medical procedure on a patient and the second medical imaging session includes performance of the second type of medical procedure on the patient.
 10. The method of claim 1, wherein the first configuration is associated with a first user profile and the second configuration is associated with a second user profile.
 11. The method of claim 10, wherein the first medical imaging session includes imaging a patient and the second medical imaging session includes imaging the patient.
 12. The method of claim 10, wherein the first configuration data and the second configuration data are both associated with the same type of medical procedure.
 13. The method of claim 1, wherein the first medical imaging session is a first surgical session and the second medical imaging session is a second surgical session.
 14. The method of claim 1, wherein the at least one medical imaging processing algorithm implemented in the second configuration comprises a smoke detection algorithm and generating the enhanced second medical imaging data comprises enhancing clarity of one or more portions of one or more images associated with smoke.
 15. The method of claim 1, wherein the first medical imaging processing algorithm is configured to detect a feature of imaged tissue.
 16. The method of claim 15, wherein the feature of imaged tissue is tissue perfusion, a location of a blood vessel, an amount of blood flow, a dimension of imaged tissue, or a combination thereof.
 17. The method of claim 1, wherein the enhanced second medical imaging data comprises an overlay on at least a portion of the second medical imaging data.
 18. The method of claim 1, wherein the reconfigurable hardware processor is reconfigured prior to a start of imaging.
 19. The method of claim 1, wherein one or more medical imaging processing algorithms are implemented in both the first and second configurations.
 20. The method of claim 1, wherein the second medical imaging data comprises at least one of video frames and an image.
 21. The method of claim 1, wherein the second medical imaging data is received from an endoscopic imaging system.
 22. The method of claim 21, wherein the second medical imaging data is received from a camera control unit.
 23. The method of claim 1, wherein the reconfigurable hardware processor is an FPGA or a GPU.
 24. The method of claim 1, comprising receiving the second medical imaging data from a first device, receiving data from a second medical device, and outputting a display feed to the display, the display feed comprising the enhanced second medical imaging data and at least a portion of the data from the second medical device.
 25. The method of claim 24, comprising receiving the second medical imaging data and the data from the second medical device at a first processor, transmitting the second medical imaging data from the first processor to the reconfigurable hardware processor, receiving the enhanced second medical imaging data from the reconfigurable hardware processor at the first processor, and generating, by the first processor, the display feed by combining the enhanced second medical imaging data with the at least a portion of the data associated with the second medical device.
 26. The method of claim 1, wherein the first configuration data is stored in a remote memory and received via a network connection.
 27. A method for displaying medical imaging data comprising: receiving first image data generated by a first medical imaging device, wherein the first image data comprises a field of view (FOV) portion and a non-FOV portion; identifying the non-FOV portion of the first image data; generating cropped first image data by removing at least a portion of the non-FOV portion of the first image data; and displaying the cropped first image data in a first portion of a display and additional information in a second portion of the display.
 28. The method of claim 27, wherein the non-FOV portion is identified using edge detection.
 29. The method of claim 28, wherein the first image data comprises a series of video frames and the edge detection is performed on more than one frame.
 30. The method of claim 27, wherein the non-FOV portion is identified using one or more of a location of a center of the FOV portion and a measurement associated with a dimension of the FOV portion.
 31. The method of claim 30, wherein the location of a center of the FOV portion and the measurement associated with a dimension of the FOV portion are determined during an imaging session initialization process.
 32. The method of claim 31, wherein the imaging session initialization process is a white balancing process.
 33. The method of claim 27, wherein the first image data comprises a rectangular image or video frame and the FOV portion is a circular portion of the rectangular image or video frame.
 34. The method of claim 27, wherein the first image data comprises a video frame.
 35. The method of claim 27, wherein the first image data is received on a first input of a medical imaging processing system and the additional information is based on data received on a second input of the medical imaging processing system.
 36. The method of claim 35, comprising transmitting a display feed from the medical imaging processing system to the display, the display feed comprising a combination of the cropped first image data and the additional information.
 37. The method of claim 27, further comprising: receiving second image data generated by a second medical imaging device; identifying a non-FOV portion of the second image data; generating cropped second image data by removing at least a portion of the non-FOV portion of the second image data; and displaying the cropped second image data in the second portion of the display.
 38. The method of claim 37, wherein the first image data is received on a first input of a medical imaging processing system and the second image data is received on a second input of the medical imaging processing system.
 39. The method of claim 38, comprising transmitting a display feed from the medical imaging processing system to the display, the display feed comprising a combination of the cropped first image data and the cropped second image data.
 40. The method of claim 27, wherein the cropped first image data and the additional information are located on the display based on configuration data stored in a memory.
 41. The method of claim 40, wherein the configuration data comprises user-specified configuration data.
 42. The method of claim 41, wherein the configuration data is received via a network connection.
 43. The method of claim 27, wherein the first image data is received from an endoscopic imaging system, an intraoperative C-arm imaging system, or an ultrasound system.
 44. The method of claim 43, wherein the first image data is received from a camera control unit.
 45. The method of claim 27, wherein the additional information comprises one or more of patient data, metrics, a graph, an image, device status, and a video feed.
 46. A reconfigurable medical imaging processing system comprising: a display; memory; a reconfigurable hardware processor; and a second processor configured for: configuring the reconfigurable hardware processor into a first configuration for a first medical imaging session based on first configuration data stored in the memory, wherein the reconfigurable hardware processor in the first configuration is configured to implement at least a first medical imaging processing algorithm and to generate enhanced first medical imaging data for display on the display at least in part by processing first medical imaging data using the first medical imaging processing algorithm, and reconfiguring the reconfigurable hardware processor into a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the reconfigurable hardware processor in the second configuration is configured to implement at least a second medical imaging processing algorithm and to generate enhanced second medical imaging data for display on the display at least in part by processing second medical imaging data using the second medical imaging processing algorithm.
 47. A system for displaying medical imaging data comprising: one or more data inputs; one or more processors; and one or more displays, wherein the one or more data inputs are configured for receiving first image data generated by a first medical imaging device, wherein the first image data comprises a field of view (FOV) portion and a non-FOV portion, and the one or more processors are configured for identifying the non-FOV portion of the first image data and generating cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and transmitting the cropped first image data for display in a first portion of the display and additional information for display in a second portion of the one or more displays. 